{{Quickfixn}} Volume of data

[James Riehl]

I don’t know what’s entailed with overwriting these methods, but it would be an interesting exercise.

Grant have you ever seen performance issues for reasons other than on message thread hogging?

Sent from my iPhone

> On Apr 23, 2018, at 10:33, Gabe Barwick <barwickhome at hotmail.com> wrote:
> 
> 
> Hi James,
> 
> Thanks for your input.
> 
> I do everything with my own code in OnMessage  on background threads.
> 
> BUT QuickFix/N writes the FIX log message and event files itself on main thread. Do you recommend overriding these as well?
> 
> 
> 
>  
> From: Quickfixn <quickfixn-bounces at lists.quickfixn.com> on behalf of James Riehl <pete.riehl at gmail.com>
> Sent: 23 April 2018 14:24
> To: Mailing list for QuickFIX/n
> Subject: Re: {{Quickfixn}} Volume of data
>  
> Your on message should do nothing other than queue processing. If your writing to disk in the onMessage then you’re doing too much. Queue up and process, I recommend blocking collection. 
> 
> Sent from my iPhone
> 
> On Apr 23, 2018, at 10:13, Gabe Barwick <barwickhome at hotmail.com> wrote:
> 
>> Hi Kunal,
>> 
>> So you think the QuickFix/N own FIX message logger is too slow even with SSD?
>> 
>> I can override it easily, but was hoping it was unnecessary - but you think it would be worthwhile?
>> 
>> 
>>  
>> From: Quickfixn <quickfixn-bounces at lists.quickfixn.com> on behalf of Kunal CHANGELA <kunal.changela at us.bnpparibas.com>
>> Sent: 23 April 2018 13:59
>> To: Mailing list for QuickFIX/n
>> Subject: Re: {{Quickfixn}} Volume of data
>>  
>> Instead of writing to disk you could write to an in-memory queue and then have another thread consume that.
>> 
>> I think you want to effectively receive the message, add to your queue and return back….dont spend time in the handler of the message to do work.
>> 
>>  
>> 
>> Thanks,
>> 
>> 
>> 
>> Kunal Changela
>> 
>> Credit Risk and PnL Development
>> 
>> Group Email: DL CRD IT NY (x841 4876, Intl: x721 4876)
>> 
>>  
>> 
>> 787 7th Avenue, New York, NY 10019
>> 
>> Tel: +1 212 841 2640
>> 
>> Mobile: +1 917 216 8852
>> 
>> kunal.changela at us.bnpparibas.com
>> 
>>  
>> 
>> <image002.gif>
>> 
>> <image003.jpg>  Do not print this document unless it is necessary, consider the environment
>> 
>>  
>> 
>> From: Quickfixn [mailto:quickfixn-bounces at lists.quickfixn.com] On Behalf Of Gabe Barwick
>> Sent: Monday, April 23, 2018 9:47 AM
>> To: quickfixn at lists.quickfixn.com
>> Subject: [EXTERNAL] {{Quickfixn}} Volume of data
>> 
>>  
>> 
>> Hi,
>> 
>>  
>> 
>> I am receiving a large amount of data (1K+ fix messages per second) from a vendor on my Windows 2008 server.
>> 
>>  
>> 
>> In OnMessage() I put a return so that I don't actually do anything with the data. All that happens is that QuickFix/N itself writes fix message to SS disk.
>> 
>>  
>> 
>> My vendor tells me that I am a slow "consumer" at peak times, meaning I am not handling messages fast enough.
>> 
>>  
>> 
>> What can I do to improve things? As I am not doing anything other than disk-write, am I correct to assume this is the problem?
>> 
>>  
>> 
>> Should I use a background writing class (on separate thread) instead of QuickFix's
>> 
>> own?
>> 
>>  
>> 
>> Any thoughts greatly appreciated
>> 
>>  
>> 
>> 
>> This message and any attachments (the "message") is intended solely for the addressees and is confidential. If you receive this message in error, please delete it and immediately notify the sender. Any use not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. The internet can not guarantee the integrity of this message. BNP PARIBAS (and its subsidiaries) shall (will) not therefore be liable for the message if modified. Please note that certain functions and services for BNP Paribas may be performed by BNP Paribas RCC, Inc.
>> 
>> 
>> Unless otherwise provided above, this message was sent by BNP Paribas, or one of its affiliates in Canada, having an office at 1981 McGill College Avenue, Montreal, QC, H3A 2W8, Canada. To the extent this message is being sent from or to Canada, you may unsubscribe from receiving commercial electronic messages by using this link: www.bnpparibas.ca/en/unsubscribe/. See www.bnpparibas.ca for more information on BNP Paribas, in Canada.
>> 
>> _______________________________________________
>> Quickfixn mailing list
>> Quickfixn at lists.quickfixn.com
>> http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com
> _______________________________________________
> Quickfixn mailing list
> Quickfixn at lists.quickfixn.com
> http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quickfixn.com/pipermail/quickfixn-quickfixn.com/attachments/20180423/1c43312e/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 11356 bytes
Desc: not available
URL: <http://lists.quickfixn.com/pipermail/quickfixn-quickfixn.com/attachments/20180423/1c43312e/attachment-0002.png>