{{Quickfixn}} Socket Deadlock Issue

[Thomas Tomiczek]

But is this not a bad pattern?

The way I handle this is:


*         Take incoming message

*         Propagate to my own queue, then handle it from there.

As a result, I am not even "cracking" messages in the receiving thread, but later in the processing handler. This one runs, as I said, in a  separate thread with an in memory queue ;)

Yes, it COULD fill up, but that would require a large amount of messages ;)

Regards

Thomas

From: quickfixn-bounces at lists.quickfixn.com [mailto:quickfixn-bounces at lists.quickfixn.com] On Behalf Of Matt Wood
Sent: 08 June 2012 16:31
To: Mailing list for QuickFIX/n
Subject: Re: {{Quickfixn}} Socket Deadlock Issue

Christian,

Hi, I'm another user and recent contributor of the quickfix/n library. I have encountered this in a related fashion (but triggered differently) and agree with your final paragraph:

"However, in the long run, we may need / want to have distinct receive and process threads. As it is, the same thread will block while calling into the actual FIX application for message processing. Should that processing take a long time or involve, for whatever reason, sending a large number of messages ... then we could wind up in a similar problem. Say the FIX application took 2 minutes to process some particular received message (in a blocking synchronous way) ... and during that 2 minutes, the other side of the FIX connection had sent enough messages to fill up it's socket buffer and block. By ensuring that message processing is in a separate thread from the socket reading then we will guarantee (in a far better way) that our socket should never wind up inadvertently blocking the other side. "

Some of my processing (mainly database updates) of messages causes a pile up when receiving an exceptionally large number of messages in a short time period. Specifically, I get this at the end of the day (around 5pm) when the system we're connected to propagates a series of IOI messages to mark market offerings as being unavailable (due to traders being forcibly logged out of the system for the day). The pile up manifests itself as "TCP ZeroWindow" messages to be sent from our side telling the foreign system to delay sending further messages. Eventually the delayed TCP messages are sent and received as our system catches up, but are rejected by our fix engine due to the fact that the messages exceed the configured latency (2 minutes!). Of course there is much I can do to alleviate this particular problem, such as creating some sort of queue and threading at a higher place in the app. However, I do feel that a distinct processing thread makes sense and would benefit the overall robustness of the application.

-Matt Wood

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quickfixn.com/pipermail/quickfixn-quickfixn.com/attachments/20120608/f2911b67/attachment-0002.htm>