{{Quickfixn}} Socket Deadlock Issue

[Christian Jungers]

Right … the issue is that this relies on the application developer to “do
the right thing” and keep their receive thread separate from their
processing thread. But even if the application does use this pattern, the
problem I’m seeing right now is that the internal QuickFIX.NET library
doesn’t follow that pattern. Thus, with the large resend request messages
being automatically / immediately processed on the receive thread, it opens
the FIX engine up to the possibility of a blocked socket.

So all I’m suggesting is to take that pattern and apply it internally to
QuickFIX as well. That way, whether or not the application developer
follows this pattern, they will be protected by a robust and scalable
QuickFIX engine.

- Christian
*
*
*
*
*
*
*Christian.Jungers at CM3.com - Chief Technology Officer** - Tel
877.263.1669 x705
- Fax 877.263.1669*



On Fri, Jun 8, 2012 at 10:56 AM, Thomas Tomiczek
<t.tomiczek at nettecture.com>wrote:

>  But is this not a bad pattern?****
>
> ** **
>
> The way I handle this is:****
>
> ** **
>
> **·         **Take incoming message****
>
> **·         **Propagate to my own queue, then handle it from there.****
>
> ** **
>
> As a result, I am not even “cracking” messages in the receiving thread,
> but later in the processing handler. This one runs, as I said, in a
> separate thread with an in memory queue ;)****
>
> ** **
>
> Yes, it COULD fill up, but that would require a large amount of messages ;)
> ****
>
> ** **
>
> Regards****
>
> ** **
>
> Thomas****
>
> ** **
>
> *From:* quickfixn-bounces at lists.quickfixn.com [mailto:
> quickfixn-bounces at lists.quickfixn.com] *On Behalf Of *Matt Wood
> *Sent:* 08 June 2012 16:31
> *To:* Mailing list for QuickFIX/n
> *Subject:* Re: {{Quickfixn}} Socket Deadlock Issue****
>
> ** **
>
> Christian,****
>
> ** **
>
> Hi, I'm another user and recent contributor of the quickfix/n library. I
> have encountered this in a related fashion (but triggered differently) and
> agree with your final paragraph:****
>
> ** **
>
> "However, in the long run, we may need / want to have distinct receive and
> process threads. As it is, the same thread will block while calling into
> the actual FIX application for message processing. Should that processing
> take a long time or involve, for whatever reason, sending a large number of
> messages ... then we could wind up in a similar problem. Say the FIX
> application took 2 minutes to process some particular received message (in
> a blocking synchronous way) ... and during that 2 minutes, the other side
> of the FIX connection had sent enough messages to fill up it's socket
> buffer and block. By ensuring that message processing is in a separate
> thread from the socket reading then we will guarantee (in a far better way)
> that our socket should never wind up inadvertently blocking the other
> side. "****
>
> ** **
>
> Some of my processing (mainly database updates) of messages causes a pile
> up when receiving an exceptionally large number of messages in a short time
> period. Specifically, I get this at the end of the day (around 5pm) when
> the system we're connected to propagates a series of IOI messages to mark
> market offerings as being unavailable (due to traders being forcibly logged
> out of the system for the day). The pile up manifests itself as "TCP
> ZeroWindow" messages to be sent from our side telling the foreign system to
> delay sending further messages. Eventually the delayed TCP messages are
> sent and received as our system catches up, but are rejected by our fix
> engine due to the fact that the messages exceed the configured latency (2
> minutes!). Of course there is much I can do to alleviate this particular
> problem, such as creating some sort of queue and threading at a higher
> place in the app. However, I do feel that a distinct processing thread
> makes sense and would benefit the overall robustness of the application.**
> **
>
> ** **
>
> -Matt Wood****
>
> ** **
>
> _______________________________________________
> Quickfixn mailing list
> Quickfixn at lists.quickfixn.com
> http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quickfixn.com/pipermail/quickfixn-quickfixn.com/attachments/20120608/39a3a0b9/attachment-0002.htm>