{{Quickfixn}} ICE Security Definition Request

[Grant Birchmeier]

Pasting my answer from the C++ list, because it's the same for any QF port:

I'm very familiar with ICE's SecDef/TCR interface, as I've maintained a
client app that supports it for a decade.

ICE's SecDef responses can be frickin' huge multipart beasts.  I suspect
you are requesting too much too fast, and then trying to handle them all on
the QF event-processing thread.  The messages are backing up in the queue,
blocking the engine from responding to ICE in a timely fashion.

*Suggestion 1:* All QF callbacks should return quickly.  If you have
expensive processing, delegate it to a separate worker thread.  In your
case: create a second worker thread to handle SecDefs.  When you receive a
SecDef, your OnMessage(SecDef) callback should push it into a queue that
the worker thread will consume from.

*Suggestion 2:* Put a delay between each SecDef Request you send.  As high
as you can tolerate.  ICE will send those responses fast and frequently;
you've gotta give your OnMessage() some room to handle them.

I have a client whose users wants to send 50+ SecDefRequests every
morning.  Right now we have them staggering each request by 60 seconds.  On
the downside, it means they have to wait an hour to get the full set.

(Some responses are bigger than others.  If you wanted, you could look at
the logs and figure out which Defs are huge and which are small, and only
do the delay after requesting big ones.)

-Grant

On Mon, Mar 2, 2020 at 11:05 AM Veduruparthi Raju <rveduruparthi at gmail.com>
wrote:

> I am developing a project in .NET to connect to ICE and i am using the
> QuickFIX engine version 1.9 and have a problem with the
> security definitions subscriptions.
>
> I am subscribing to the security definitions and am receiving the
> responses (35=d) and in the onMessage method, i straight away processing
> the responses and writing them to the database directly. The problem is
> every 2 minutes my sessions logs out with the below error message. The
> connection forcibly closes by ICE.
>
> I have contacted them and were saying to increase the heap size which i
> was not convinced and need help to understand what is the correct way of
> handling this in .NET since i know the market reference data is huge.
>
> Appreciate your response.
>
> System.IO.IOException: Unable to write data to the transport connection:
> An existing connection was forcibly closed by the remote host. --->
> System.Net.Sockets.SocketException: An existing connection was forcibly
> closed by the remote host
>    at System.Net.Sockets.NetworkStream.Write(Byte[] buffer, Int32 offset,
> Int32 size)
>    --- End of inner exception stack trace ---
>    at System.Net.Sockets.NetworkStream.Write(Byte[] buffer, Int32 offset,
> Int32 size)
>    at QuickFix.SocketInitiatorThread.Send(String data)
>    at QuickFix.Session.Send(String message)
>    at QuickFix.Session.SendRaw(Message message, Int32 seqNum)
>    at QuickFix.Session.Next()
>    at QuickFix.Session.Next(MessageBuilder msgBuilder)
>    at QuickFix.SocketInitiatorThread.ProcessStream()
>    at QuickFix.SocketInitiatorThread.Read()
>
> Thanks,
> _______________________________________________
> Quickfixn mailing list
> Quickfixn at lists.quickfixn.com
> http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com
>


-- 
Grant Birchmeier
*Connamara Systems, LLC*
*Made-To-Measure Trading Solutions.*
Exactly what you need. No more. No less.
http://connamara.com

-- 
This email, along with any attachments, is confidential. If you believe you 
received this message in error, please contact the sender immediately and 
delete all copies of the message. Thank you from Connamara Systems, LLC.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quickfixn.com/pipermail/quickfixn-quickfixn.com/attachments/20200302/753ed189/attachment.htm>