{{Quickfixn}} FW: Execution report not coming for few trades

Lanfranco Morini Lanfranco.Morini at cegeka.it
Fri Sep 25 02:19:36 PDT 2020 

Rajkumar,
keep in mind that if they sent an Execution Report with a MsgSeqNum (tag 34) and a SendingTime (tag 52) and you didn’t receive it, at the next MsgSeqNum received, noting the missing sequence number, your FIX Engine should have sent automatically a ResendRequest message…

Summarizing: the cases are two:
1st they didn’t actually send any ER,
2nd they sent it and you received it (and, as per Grant email, you should find it in your logs), but you lost it during processing FromApp callback…

Best
Lan


Lanfranco Giuseppe Morini
Program Manager
Banking & Finance
APPLICATIONS

E  Lanfranco.Morini at cegeka.it<mailto:Lanfranco.Morini at cegeka.it>
T +39 02 254427 1
M +39 347 7282717

[cid:image003.png at 01D6932C.0886BBA0]
CEGEKA Via A. Volta 16
20093 Cologno Monzese MI, Italy
T +39 02 254 427 1
WWW.CEGEKA.IT<http://www.cegeka.it/>
[cid:image004.jpg at 01D6932C.0886BBA0]<https://www.linkedin.com/company/cegeka_italia>
[cid:image005.jpg at 01D6932C.0886BBA0]<https://twitter.com/Cegeka_IT>
[cid:image006.jpg at 01D6932C.0886BBA0]<https://www.facebook.com/Cegeka.Italia>
[cid:image007.jpg at 01D6932C.0886BBA0]<https://www.youtube.com/channel/UCyK4GfmaAT3MEJttkpfWoyQ>


From: Quickfixn <quickfixn-bounces at lists.quickfixn.com> On Behalf Of Grant Birchmeier
Sent: giovedì 24 settembre 2020 15:58
To: Mailing list for QuickFIX/n <quickfixn at lists.quickfixn.com>
Cc: Arun Sivaprakash <arunsivaprakash at gmail.com>
Subject: Re: {{Quickfixn}} FW: Execution report not coming for few trades

Did their logs show that they sent an Execution report?

If yes, what did your logs show at the same time?

On Thu, Sep 24, 2020 at 12:34 AM Rajkumar R <kraj232 at gmail.com<mailto:kraj232 at gmail.com>> wrote:
Hi grant,
We have requested the FIX logs from FXALL (counter party) and we checked the same, they confirmed that the trades has been sent, recently we have moved our service to a new server and we faced the same issue again. Any help would be appreciated, any code change kindly let us know.

Awaiting your reply

With regards,
R. Rajkumar

On Thu, 3 Sep, 2020, 1:55 PM Rajkumar R, <kraj232 at gmail.com<mailto:kraj232 at gmail.com>> wrote:
Hi Grant,

I have just shared my  code with you just only for your reference ,  since you suspect that  QF OnMessage callbacks might be an issue. Our volume of trades per day is not too high. if any code change needs  to undergo a big approval process and it is quite time consuming as well , so we are looking for some configuration change if any.

We are analyzing  the network  packets ,firewall of ours as well counterparty(FXALL).

Thanks for the C# reference

With Regards,
R.Rajkumar

On Thu, Sep 3, 2020 at 9:00 AM Rajkumar R <kraj232 at gmail.com<mailto:kraj232 at gmail.com>> wrote:
Hi grant,

Thanks for your help and support.

With regards,
R. Rajkumar

On Wed, 2 Sep, 2020, 11:18 PM Grant Birchmeier, <gbirchmeier at connamara.com<mailto:gbirchmeier at connamara.com>> wrote:
Rajkumar,

We're not just gonna be able to give you the magic answer.  You're going to have to analyze your own code.  You should be able to know if you're performing expensive operations (DB lookups and writes maybe?) inside your OnMessage implementation.

If it turns out that I am correct, then you will need to implement a workqueue: Your OnMessage should push the msg into the queue and immediately return; a SEPARATE THREAD will read messages off the queue and perform the expensive operations that your app needs.

I don't have an example handy, but I believe you might want to read about the C# BlockingCollection and ConcurrentQueue.

-Grant



On Wed, Sep 2, 2020 at 10:00 AM Rajkumar R <kraj232 at gmail.com<mailto:kraj232 at gmail.com>> wrote:
Hi Lanfranco,

Thanks for the clarification. Attached code snippet which i used in OnMessage event.


With Regards.
R.Rajkumar





On Wed, Sep 2, 2020 at 7:04 PM Lanfranco Morini <Lanfranco.Morini at cegeka.it<mailto:Lanfranco.Morini at cegeka.it>> wrote:
Yes, it’s another possibility...
In this case Rajkumar should decouple the OnMessage callbacks with their implementation using an internal layer of queues…

Lan

From: Quickfixn <quickfixn-bounces at lists.quickfixn.com<mailto:quickfixn-bounces at lists.quickfixn.com>> On Behalf Of Grant Birchmeier
Sent: mercoledì 2 settembre 2020 15:28
To: Mailing list for QuickFIX/n <quickfixn at lists.quickfixn.com<mailto:quickfixn at lists.quickfixn.com>>
Cc: Arun Sivaprakash <arunsivaprakash at gmail.com<mailto:arunsivaprakash at gmail.com>>
Subject: Re: {{Quickfixn}} FW: Execution report not coming for few trades

I suspect that Rajkumar's QF OnMessage callbacks are doing too much work and taking too much time to return.

Messages in QF are processed serially; message-2 will not be processed until OnMessage(message-1) returns.

If OnMessage takes too long to return, the receiving queue will start to back up during heavy traffic.  This would result in TestRequests and connection drops, as the counterparty tries to deal with your lagginess.

On Wed, Sep 2, 2020 at 5:20 AM Lanfranco Morini <Lanfranco.Morini at cegeka.it<mailto:Lanfranco.Morini at cegeka.it>> wrote:
Hi Rajkumar,
I’m quite sure that no bugs are affecting your QuickFIX/n.

I can add some further considerations to this premise (which surely will not be exhaustive)

  1.  A Logon message with 141=Y during the trading day is very dangerous: after this kind of logon, you (and your counterparty) “lose” every application message sent before (read: a resend request for some execution reports exchanged before the new logon will not returns anything, because both of FIX Engines erase every sequence numbers and their related messages)
  2.  From the messages log that you sent (even if partial), it can be deducted that the disconnection comes from a network failure (no test request, nor logout messages are in your log)
  3.  FXALL seems to have some problem in managing your messages, in fact they sent a test request (35=1) at 20200828-13:29:53.409 (you registered it at 20200828-13:30:15.911) even if you sent a previous message at 52=20200828-13:29:57.833 (less than 60 seconds configured as HeartBeat interval)

     *   Another alternative can be that there are some latency on the network (it can also cause a forced closure of the network connection between you and FXALL): this can be deducted looking at the timestamp differences in several messages

example

20200828-13:28:42.130 : 8=FIX.4.4  9=74  35=A  34=1  49=xxxxxxx  52=20200828-13:28:42.130  56=FXALL  98=0  108=60  141=Y  10=200

20200828-13:28:43.005 : 8=FIX.4.4  9=74  35=A  49=FXALL  56=xxxxxxx  34=1  52=20200828-13:28:42.963  108=60  141=Y  98=0  10=214

From this two logon message we can note that comparing tag 52 timestamp and the log time stamp (written before the message) you received the FXALL message at least after 42ms they sent it 13:28:43.963 vs. 13:28:43.005

But if we look at the test request sent by FXALL you received it after 22 seconds they sent it

20200828-13:30:15.911 : 8=FIX.4.4  9=78  35=1  49=FXALL  56=xxxxxxx  34=8  52=20200828-13:29:53.409  112=20200828-13:29:53  10=198

We can say that changing the 141=1 settings can help to better understand what is causing the disconnection and probably the loss of messages, in fact without a forced sequence reset during logon, an automatic alignment of the lost messages should happen after logon…

Last but not least: if FXALL is sure that they sent the execution reports that you miss, they should be able to provide you their MsgSeqNum: in this case you are able to look for them in your log and verifiy if really lost themo or if they never arrived\never been sent

I wait for your feedback related to my considerations,

Best regards,
Lanfranco


From: Quickfixn <quickfixn-bounces at lists.quickfixn.com<mailto:quickfixn-bounces at lists.quickfixn.com>> On Behalf Of Rajkumar R
Sent: mercoledì 2 settembre 2020 09:35
To: Mailing list for QuickFIX/n <quickfixn at lists.quickfixn.com<mailto:quickfixn at lists.quickfixn.com>>
Cc: Arun Sivaprakash <arunsivaprakash at gmail.com<mailto:arunsivaprakash at gmail.com>>
Subject: Re: {{Quickfixn}} FW: Execution report not coming for few trades

Hi LanFranco,

Thanks for your concern.

Below attached files are
1.FIX Connectivity (FIX.4.4-olam-eur-FXALL.event.current)
2.FIX Message Logs ( FIX.4.4-olam-eur-FXALL.messages.current) 3.configuration file (QfSettings.txt)

The message logs is extracted based on the event logs disconnectivity timings

Note : I have already shared the event & message on  28-Aug-2020

If any more inputs required do please let me know

Thanks in advance

With Regards,
R.Rajkumar



On Mon, Aug 31, 2020 at 9:58 PM Lanfranco Morini <Lanfranco.Morini at cegeka.it<mailto:Lanfranco.Morini at cegeka.it>> wrote:
Rajkumar,
we never received the messages log that you wrote you have sent, and also config file never arrive: without messages it is impossible to understand what went wrong on your Fix Engine

That’s why Grant said that is impossible to help you: FIX Protocol is message based and each message is the consequence of the previous one, we need them to understand if you missed\discard the execution reports or if your counterparty never sent (even if they say the opposite)…

Hope it can help…

Lanfranco


From: Quickfixn <quickfixn-bounces at lists.quickfixn.com<mailto:quickfixn-bounces at lists.quickfixn.com>> On Behalf Of Rajkumar R
Sent: lunedì 31 agosto 2020 17:45
To: Mailing list for QuickFIX/n <quickfixn at lists.quickfixn.com<mailto:quickfixn at lists.quickfixn.com>>
Subject: Re: {{Quickfixn}} Execution report not coming for few trades

Hi,

Thanks for your help

With regards,
R. Rajkumar
On Mon, 31 Aug, 2020, 8:51 PM Grant Birchmeier, <gbirchmeier at connamara.com<mailto:gbirchmeier at connamara.com>> wrote:
No, we can't.

The information that you've provided is not sufficient, and my attempts to get better info out of you have failed.

You didn't try the actions that I suggested.   You didn't provide the info that I requested.  Your last email did not make any sense.

If you want help, re-read my last email and do what I have asked.

-Grant




On Mon, Aug 31, 2020 at 10:10 AM Rajkumar R <kraj232 at gmail.com<mailto:kraj232 at gmail.com>> wrote:
Hi,

Can anybody help please.

With regards,
R. Rajkumar

On Fri, 28 Aug, 2020, 10:27 PM Rajkumar R, <kraj232 at gmail.com<mailto:kraj232 at gmail.com>> wrote:
Hi,
Am sorry in my previous mail I have taken a wrong date of current and message logs, that is the reason I have resent the 18jul2020 logs of current and message log which match the timestamps. Just to have clarity I have added the config file as well.

With regards,
Rajkumar



Regards,
Rajkumar

On Fri, 28 Aug, 2020, 9:07 PM Grant Birchmeier, <gbirchmeier at connamara.com<mailto:gbirchmeier at connamara.com>> wrote:
With all due respect, I get the impression that you don't really know what you're doing.

It's like you're sending me pastes of errors that you aren't even reading.  They have nothing to do with app-layer messaging.  Do these timestamps even correspond with any ExecutionReports that you're supposedly missing?

You're sending logs for connection drops.  I already told you what I think of these.  I think they have nothing to do with your missed messages.

Do this:

  1.  Ask your counterparty for a message that they sent that you missed.
  2.  Go in your message log, find that time, and look at what's happening.
If you wish to ask for further help, please send details about (1) and (2).

-Grant
_______________________________________________
Quickfixn mailing list
Quickfixn at lists.quickfixn.com<mailto:Quickfixn at lists.quickfixn.com>
http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com
_______________________________________________
Quickfixn mailing list
Quickfixn at lists.quickfixn.com<mailto:Quickfixn at lists.quickfixn.com>
http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com


--
Grant Birchmeier
Connamara Systems, LLC
Made-To-Measure Trading Solutions.
Exactly what you need. No more. No less.
http://connamara.com

This email, along with any attachments, is confidential. If you believe you received this message in error, please contact the sender immediately and delete all copies of the message. Thank you from Connamara Systems, LLC.
_______________________________________________
Quickfixn mailing list
Quickfixn at lists.quickfixn.com<mailto:Quickfixn at lists.quickfixn.com>
http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com
_______________________________________________
Quickfixn mailing list
Quickfixn at lists.quickfixn.com<mailto:Quickfixn at lists.quickfixn.com>
http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com


--
Grant Birchmeier
Connamara Systems, LLC
Made-To-Measure Trading Solutions.
Exactly what you need. No more. No less.
http://connamara.com

This email, along with any attachments, is confidential. If you believe you received this message in error, please contact the sender immediately and delete all copies of the message. Thank you from Connamara Systems, LLC._______________________________________________
Quickfixn mailing list
Quickfixn at lists.quickfixn.com<mailto:Quickfixn at lists.quickfixn.com>
http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com
_______________________________________________
Quickfixn mailing list
Quickfixn at lists.quickfixn.com<mailto:Quickfixn at lists.quickfixn.com>
http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com


--
Grant Birchmeier
Connamara Systems, LLC
Made-To-Measure Trading Solutions.
Exactly what you need. No more. No less.
http://connamara.com

This email, along with any attachments, is confidential. If you believe you received this message in error, please contact the sender immediately and delete all copies of the message. Thank you from Connamara Systems, LLC.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quickfixn.com/pipermail/quickfixn-quickfixn.com/attachments/20200925/5911f37f/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 8099 bytes
Desc: image001.jpg
URL: <http://lists.quickfixn.com/pipermail/quickfixn-quickfixn.com/attachments/20200925/5911f37f/attachment-0012.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 8483 bytes
Desc: image002.jpg
URL: <http://lists.quickfixn.com/pipermail/quickfixn-quickfixn.com/attachments/20200925/5911f37f/attachment-0013.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 4821 bytes
Desc: image003.png
URL: <http://lists.quickfixn.com/pipermail/quickfixn-quickfixn.com/attachments/20200925/5911f37f/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.jpg
Type: image/jpeg
Size: 1224 bytes
Desc: image004.jpg
URL: <http://lists.quickfixn.com/pipermail/quickfixn-quickfixn.com/attachments/20200925/5911f37f/attachment-0014.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image005.jpg
Type: image/jpeg
Size: 1119 bytes
Desc: image005.jpg
URL: <http://lists.quickfixn.com/pipermail/quickfixn-quickfixn.com/attachments/20200925/5911f37f/attachment-0015.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.jpg
Type: image/jpeg
Size: 1166 bytes
Desc: image006.jpg
URL: <http://lists.quickfixn.com/pipermail/quickfixn-quickfixn.com/attachments/20200925/5911f37f/attachment-0016.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image007.jpg
Type: image/jpeg
Size: 1145 bytes
Desc: image007.jpg
URL: <http://lists.quickfixn.com/pipermail/quickfixn-quickfixn.com/attachments/20200925/5911f37f/attachment-0017.jpg>

More information about the Quickfixn mailing list