{{Quickfixn}} Quickfixn Digest, Vol 7, Issue 13

Thu May 17 01:30:11 PDT 2012

Stupid question...

Do you have enough bandwidth?

MarketData in FIX - without FAST - is extremely inefficient network bandwidth wise. 1000 symbols is also not something the other side may be prepared to send (i.e. have you validated that this does not overload the other end simply)?

I personally track around 200.000 symbols but - I do NOT do that with FIX, I have a high end data feed coming in. 1000 symbols via FIX on an internet style connection is a disaster waiting to happen. Better with FAST, but then both sides need to support it.

I know a lot of endpoints that would simply not handle 10.000 market data requests well.

Regards

Thomas

-----Original Message-----
From: quickfixn-bounces at lists.quickfixn.com [mailto:quickfixn-bounces at lists.quickfixn.com] On Behalf Of Felipe Soares
Sent: 17 May 2012 10:28
To: quickfixn at lists.quickfixn.com
Subject: Re: {{Quickfixn}} Quickfixn Digest, Vol 7, Issue 13

Maybe the problem is that inside my onmessage securitydefinition (response) i have a session.sendtotarget, requesting marketdata. That processo happens 10000 times. Is this wrong that i am doing?

Regards

Enviado via iPhone

Em 16/05/2012, às 19:15, quickfixn-request at lists.quickfixn.com escreveu:

> Send Quickfixn mailing list submissions to
>    quickfixn at lists.quickfixn.com
> 
> To subscribe or unsubscribe via the World Wide Web, visit
>    http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com
> or, via email, send a message with subject or body 'help' to
>    quickfixn-request at lists.quickfixn.com
> 
> You can reach the person managing the list at
>    quickfixn-owner at lists.quickfixn.com
> 
> When replying, please edit your Subject line so it is more specific 
> than "Re: Contents of Quickfixn digest..."
> 
> 
> Today's Topics:
> 
>   1.  Message Limits (Felipe Soares)
>   2. Re:  Message Limits (Mike Gatny)
> 
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Wed, 16 May 2012 16:48:19 -0300
> From: Felipe Soares <felipe.soares at traderdata.com.br>
> To: quickfixn at lists.quickfixn.com
> Subject: {{Quickfixn}} Message Limits
> Message-ID: <719684e7cfc8ceb654a1af218f9efee8 at mail.gmail.com>
> Content-Type: text/plain; charset="windows-1252"
> 
> Hi,
> 
> 
> 
> I am new to quickfix/n and having some doubts about it:
> 
> 
> 
> -          Message size limt, is there any size limit? For instance for
> sending securityDefinitiion should I send one by one? Or with a group?
> 
> -          After I receive all my securitites I send a message with
> MarketDataRequest for each symbol in a separate message.
> 
> 
> 
> For some reason my client hangs at some point?nothing happens after it 
> hanged?I can?t get the session to heartbeat?.
> 
> 
> 
> I am sending something around 10000 requests each one in one 
> message?.is this a problem?
> 
> 
> 
> Regards
> 
> Felipe
> -------------- next part -------------- An HTML attachment was 
> scrubbed...
> URL: 
> <http://lists.quickfixn.com/pipermail/quickfixn-quickfixn.com/attachme
> nts/20120516/912fc213/attachment.html>
> 
> ------------------------------
> 
> Message: 2
> Date: Wed, 16 May 2012 15:12:51 -0500
> From: Mike Gatny <mgatny at connamara.com>
> To: "Mailing list for QuickFIX/n" <quickfixn at lists.quickfixn.com>
> Subject: Re: {{Quickfixn}} Message Limits
> Message-ID:
>    
> <CAA1-azN-b=+XM2ywCVwrZBKqrpZKOjUpCxah4CcRrjhExb7Htw at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
> 
> On Wed, May 16, 2012 at 2:48 PM, Felipe Soares < 
> felipe.soares at traderdata.com.br> wrote:
> 
>> Message size limt, is there any size limit? For instance for sending 
>> securityDefinitiion should I send one by one? Or with a group?
>> 
> 
> 
>> After I receive all my securitites I send a message with 
>> MarketDataRequest for each symbol in a separate message.
>> 
> 
> The standard FIX SecurityDefinitionRequest and MarketDataRequest 
> messages allow you to request multiple symbols in one message (one per 
> group).  Your counterparty may not use plain vanilla FIX 
> messages/semantics, of course, so check their documentation.  From 
> experience, I would say it is probably not practical to request 10000 symbols in one message (or on one Session).
>  I don't think there is any hard limit in qf/n that will stop you from 
> trying, but even if you successfully subscribe to 10000 symbols, you 
> are probably going to experience performance issues trying to handle 
> that much data on one Session.  And naturally, the more heavily 
> quoted/traded the symbols are, the more messages you will have, and 
> hence more potential for performance problems.
> 
> Note that when you are handling a FIX message, i.e. during the
> fromApp()/onMessage() callbacks, you are on the qf/n thread and thus 
> blocking qf/n from handling the next message.  So you want to return 
> from
> fromApp()/onMessage() as quickly as possible so that qf/n can continue 
> processing.  If you are doing a lot of work in fromApp()/onMessage() 
> or making blocking calls there, consider queuing the work onto a 
> separate thread so that you can return more quickly.  Also, if 
> possible, consider setting up more than one Session with your 
> counterparty so that you can split up the 10000 symbols across 
> multiple Sessions (and thus maybe even multiple applications).  This 
> way, you give yourself the ability to configure your way out of 
> performance issues.  For example with this design, if you realize that 
> a handful of symbols are the most heavily quoted/traded, you can move 
> them to a separate Session, maybe even on a different server.
> 
> --
> Mike Gatny
> Connamara Systems, LLC
> -------------- next part -------------- An HTML attachment was 
> scrubbed...
> URL: 
> <http://lists.quickfixn.com/pipermail/quickfixn-quickfixn.com/attachme
> nts/20120516/cd8d222e/attachment.htm>
> 
> ------------------------------
> 
> _______________________________________________
> Quickfixn mailing list
> Quickfixn at lists.quickfixn.com
> http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com
> 
> 
> End of Quickfixn Digest, Vol 7, Issue 13
> ****************************************
_______________________________________________
Quickfixn mailing list
Quickfixn at lists.quickfixn.com
http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com