Hello, my application is used to receive us markets quotes, parsing various fids and write them into a internal round buffer. When it collects enough records then write to log file in batch. But occasionally, it will hang as shown below.
Thread 9 is a message decoder thread, it will both decode chain & ric Refresh/Update message. See from its stacktrace from gdb, it blocks at OmmConsumerImpl::registerClient when using mutex lock. I also have the same application running in normal mode, parsing fids and write to kafka but it will not block. So what's the possible reason for the hang problem ?
Is there any memory corruption from the logging thread that cause decoder thread hang ? Thanks.
(gdb) thread 9 [Switching to thread 9 (Thread 0x7f134de34700 (LWP 21627))] #0 0x00007f135f2c150d in __lll_lock_wait () from /lib64/libpthread.so.0 (gdb) bt #0 0x00007f135f2c150d in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x00007f135f2bce76 in _L_lock_941 () from /lib64/libpthread.so.0 #2 0x00007f135f2bcd6f in pthread_mutex_lock () from /lib64/libpthread.so.0 #3 0x0000000000509e5c in thomsonreuters::ema::access::OmmConsumerImpl::registerClient(thomsonreuters::ema::access::ReqMsg const&, thomsonreuters::ema::access::OmmConsumerClient&, void*, unsigned long long) () #4 0x0000000000496239 in ChainHandler::batch_register (this=0x495af00, ema_codes=...) at server/emaparser/chain_handler_base.cpp:627 #5 0x0000000000496a67 in ChainHandler::single_register (this=0x495af00, ema_code=...) at server/emaparser/chain_handler_base.cpp:684 #6 0x000000000049549b in ChainHandler::ProcessChainUpdateMessage (this=0x495af00, updateMsg=..., chain_type=43) at server/emaparser/chain_handler_base.cpp:541 #7 0x00000000004d206e in EmaDecoderMgr::DecodeMsg (this=0x232a8b0 <EmaDecoderMgr::merger_mgr_>, queue_index=0) at server/emaparser/decoder_mgr.cpp:125 #8 0x00000000004d19f4 in DecodeMsgThreadFunc (queue_index=0, arg=0x0, length=0x0) at server/emaparser/decoder_mgr.cpp:11 #9 0x0000000000970bd2 in Thread::LoopThreadFunc (arg=0x49569c0) at comm/util/thread.cpp:147 #10 0x00007f135f2bae65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007f135e8d588d in clone () from /lib64/libc.so.6