Thijs van Dien

The one thing I'm still waiting for is per project packages/components that are automatically (un)loaded by the IDE, right from the code repository they're used in (i.e. machine independent). I can't believe we still don't have something like that after so many years.

So now the DocWiki has been offline again for several days already? Impressive...

If most of all you want to prevent repeated handshakes, that's just a matter of using persistent HTTP connections (keep-alive)...? Another good option could be a RESTful batch API, which in the end helps you optimize more (e.g. database queries).

Should you go for the WebSocket idea, the closest thing to a standard is JSON-RPC 2.0, I'd say. There are ready clients available for several languages. Depending on the amount of methods you'll support, consider providing a machine readable spec using OpenRPC.

Looking forward to the PDF version. :)

Offtopic perhaps, but I'd advise you not to put such personal information on the internet like this.

Alright, a library I recently started using more does indeed leave the rounding mode changed sometimes. It's a sad reality, but I'll be (even) more suspicious of third party code and try to make the software increasingly resilient on this front over time. For now, I hope removing a major cause will solve the issues experienced. Thanks!

What would your suggestion be to avoid this situation of (sudden) undesirable results in a big legacy application where I can't just go and change all the code?

Me. 😉

This software has never experienced a problem like this before in over 20 years, so my first impulse is to track down what started causing this mess. I'm just not sure how if it I can't consistently reproduce it.

Reading a little bit more about the topic, for example here, makes me rather hopeless about it all. I wonder what the sane approach would be if I want all my "own" rounding to behave like rmNearest. I could replace all calls to Round by a custom function, but that leaves a ton of libraries like FastReport potentially doing the wrong thing. Calling SetRoundMode in a timer is about as ugly as it gets...

Check. I could extend the logging with a call to Get8087CW  to see if that's the reason. If it is, how should I go about finding out how it ends up in the 'wrong' state? 

(Cross posting from StackOverflow.)

I'm facing the bizarre situation that the same program on the same machine doing a Round() of the same floating point value does not always give the same result. At first I thought it has to be glitch because of a bit flip or something, but it keeps on coming back. Now it started to happen on a completely different machine as well. Bad results come up rather rarely, in the order of once a week. When they do, it seems they keep happening until the program is restarted. That might be a coincidence, however. So far, I've been unable to reproduce it at will. This is all happening on the main thread, by the way.

A check I consider adding is whether Round keeps pointing to the same memory address. Any other ideas?

2020-12-28 08:30:19.411 DBGrid1.Fields[17].AsString: 0,239999994635582
2020-12-28 08:30:19.411 FloatToStr(DBGrid1.Fields[17].AsFloat): 0,239999994635582
2020-12-28 08:30:19.411 FloatToStr(1000 * DBGrid1.Fields[17].AsFloat): 239,999994635582
2020-12-28 08:30:19.411 IntToStr(Round(1000 * DBGrid1.Fields[17].AsFloat)): 239
2020-12-28 08:30:19.411 FloatToStr(DBGrid1.Fields[17].AsSingle): 0,239999994635582
2020-12-28 08:30:19.411 FloatToStr(1000 * DBGrid1.Fields[17].AsSingle): 239,999994635582
2020-12-28 08:30:19.411 IntToStr(Round(1000 * DBGrid1.Fields[17].AsSingle)): 239
2020-12-28 08:30:19.411 FloatToStr(DBGrid1.Fields[17].AsExtended): 0,239999994635582
2020-12-28 08:30:19.412 FloatToStr(1000 * DBGrid1.Fields[17].AsExtended): 239,999994635582
2020-12-28 08:30:19.412 IntToStr(Round(1000 * DBGrid1.Fields[17].AsExtended)): 239
2020-12-28 08:30:19.412 CurrToStr(DBGrid1.Fields[17].AsCurrency): 0,2399
2020-12-28 08:30:19.412 CurrToStr(1000 * DBGrid1.Fields[17].AsCurrency): 239,9
2020-12-28 08:30:19.412 IntToStr(Round(1000 * DBGrid1.Fields[17].AsCurrency)): 239
2020-12-28 08:30:19.412 FloatToStr(Query1.FieldByName('FIELD').AsFloat): 0,239999994635582
2020-12-28 08:30:19.412 FloatToStr(1000 * Query1.FieldByName('FIELD').AsFloat): 239,999994635582
2020-12-28 08:30:19.412 IntToStr(Round(1000 * Query1.FieldByName('FIELD').AsFloat)): 239
2020-12-28 08:30:19.412 FloatToStr(Query1.FieldByName('FIELD').AsSingle): 0,239999994635582
2020-12-28 08:30:19.412 FloatToStr(1000 * Query1.FieldByName('FIELD').AsSingle): 239,999994635582
2020-12-28 08:30:19.413 IntToStr(Round(1000 * Query1.FieldByName('FIELD').AsSingle)): 239
2020-12-28 08:30:19.413 FloatToStr(Query1.FieldByName('FIELD').AsExtended): 0,239999994635582
2020-12-28 08:30:19.413 FloatToStr(1000 * Query1.FieldByName('FIELD').AsExtended): 239,999994635582
2020-12-28 08:30:19.413 IntToStr(Round(1000 * Query1.FieldByName('FIELD').AsExtended)): 239
2020-12-28 08:30:19.413 CurrToStr(Query1.FieldByName('FIELD').AsCurrency): 0,2399
2020-12-28 08:30:19.413 CurrToStr(1000 * Query1.FieldByName('FIELD').AsCurrency): 239,9
2020-12-28 08:30:19.413 IntToStr(Round(1000 * Query1.FieldByName('FIELD').AsCurrency)): 239
2020-12-28 08:30:19.413 CurrToStr(Query1.FieldByName('FIELD').AsCurrency): 0,2399
2020-12-28 08:30:19.413 CurrToStr(1000 * Query1.FieldByName('FIELD').AsCurrency): 239,9
2020-12-28 08:30:19.413 IntToStr(Round(1000 * Query1.FieldByName('FIELD').AsCurrency)): 239
2020-12-28 08:30:19.414 BinToHex(DBGrid1.Fields[17].AsFloat): 00000000008FC2F5FC3F

2020-12-28 08:40:46.461 DBGrid1.Fields[17].AsString: 0,239999994635582
2020-12-28 08:40:46.462 FloatToStr(DBGrid1.Fields[17].AsFloat): 0,239999994635582
2020-12-28 08:40:46.463 FloatToStr(1000 * DBGrid1.Fields[17].AsFloat): 239,999994635582
2020-12-28 08:40:46.463 IntToStr(Round(1000 * DBGrid1.Fields[17].AsFloat)): 240
2020-12-28 08:40:46.463 FloatToStr(DBGrid1.Fields[17].AsSingle): 0,239999994635582
2020-12-28 08:40:46.463 FloatToStr(1000 * DBGrid1.Fields[17].AsSingle): 239,999994635582
2020-12-28 08:40:46.463 IntToStr(Round(1000 * DBGrid1.Fields[17].AsSingle)): 240
2020-12-28 08:40:46.463 FloatToStr(DBGrid1.Fields[17].AsExtended): 0,239999994635582
2020-12-28 08:40:46.463 FloatToStr(1000 * DBGrid1.Fields[17].AsExtended): 239,999994635582
2020-12-28 08:40:46.463 IntToStr(Round(1000 * DBGrid1.Fields[17].AsExtended)): 240
2020-12-28 08:40:46.463 CurrToStr(DBGrid1.Fields[17].AsCurrency): 0,24
2020-12-28 08:40:46.463 CurrToStr(1000 * DBGrid1.Fields[17].AsCurrency): 240
2020-12-28 08:40:46.463 IntToStr(Round(1000 * DBGrid1.Fields[17].AsCurrency)): 240
2020-12-28 08:40:46.463 FloatToStr(Query1.FieldByName('FIELD').AsFloat): 0,239999994635582
2020-12-28 08:40:46.463 FloatToStr(1000 * Query1.FieldByName('FIELD').AsFloat): 239,999994635582
2020-12-28 08:40:46.463 IntToStr(Round(1000 * Query1.FieldByName('FIELD').AsFloat)): 240
2020-12-28 08:40:46.463 FloatToStr(Query1.FieldByName('FIELD').AsSingle): 0,239999994635582
2020-12-28 08:40:46.464 FloatToStr(1000 * Query1.FieldByName('FIELD').AsSingle): 239,999994635582
2020-12-28 08:40:46.464 IntToStr(Round(1000 * Query1.FieldByName('FIELD').AsSingle)): 240
2020-12-28 08:40:46.464 FloatToStr(Query1.FieldByName('FIELD').AsExtended): 0,239999994635582
2020-12-28 08:40:46.464 FloatToStr(1000 * Query1.FieldByName('FIELD').AsExtended): 239,999994635582
2020-12-28 08:40:46.464 IntToStr(Round(1000 * Query1.FieldByName('FIELD').AsExtended)): 240
2020-12-28 08:40:46.464 CurrToStr(Query1.FieldByName('FIELD').AsCurrency): 0,24
2020-12-28 08:40:46.464 CurrToStr(1000 * Query1.FieldByName('FIELD').AsCurrency): 240
2020-12-28 08:40:46.464 IntToStr(Round(1000 * Query1.FieldByName('FIELD').AsCurrency)): 240
2020-12-28 08:40:46.464 CurrToStr(Query1.FieldByName('FIELD').AsCurrency): 0,24
2020-12-28 08:40:46.464 CurrToStr(1000 * Query1.FieldByName('FIELD').AsCurrency): 240
2020-12-28 08:40:46.465 IntToStr(Round(1000 * Query1.FieldByName('FIELD').AsCurrency)): 240
2020-12-28 08:40:46.465 BinToHex(DBGrid1.Fields[17].AsFloat): 00000000008FC2F5FC3F

So far, I've been recreating my HTTP client (and SSL handler) for every request, because I'm a little paranoid of leftover state that could affect stability and/or correctness. The price I pay for that, is that I can't benefit from keep-alive. What would I have to do to make sure that no previous request has any influence on the (observable) state of TIdHTTP, except for keeping the connection open? Completely erasing the last response (how?) would be a good start, but perhaps there's more I can do to keep state to a minimum, and always start fresh to the extent possible, while being more efficient at the network level.

I am missing the point of this discussion. As has been said, it is impossible to foresee all states the service could end up in. When it appears that things are not working correctly, the priority is to get back into a known good state. Terminating the service and having it restarted by the service manager (Windows) is a means to do so. Yes, it should be logged and investigated later to hopefully prevent it in the future, but some incidents are rare enough that dealing with them in this way is acceptable. If you disagree with that, go tell the Erlang people they got it all wrong. I'm just looking for the best way to implement it.

Although I still feel that the stronger isolation of processes might be desirable, for now I am going to count on threads being good enough. Here's how my dead man's switch looks right now: when the service is started, I use CreateTimerQueueTimer with a callback that the worker is supposed to continuously prevent from executing by means of ChangeTimerQueueTimer. If it does go off, that first sets an event to give the main thread (service loop) a chance to log what's happening and exit relatively cleanly. If that doesn't happen within 5 seconds, because apparently it got into a faulty state as well, ExitProcess(1) is called. My assumption is that because this third thread only calls three "cheap" Windows API's (SetEvent, Sleep, and ExitProcess) directly, things would have to be very, very bad for that to fail. And short of some accidental denial of service attack on the thread pool, I can't really think of anything that would keep it from running at all.

If the worker were known to be well-behaved, there would be no need for such monitoring to begin with. There's networking, third party libraries, DLLs and what not. I can't know exactly what could cause a freeze; only that it will be rare, and restarting will be an effective way to get on with life. Sort of the 'Let It Crash' philosophy. I want to treat the worker as a blackbox. The only requirement is it will regularly report that it's still functional, and if it doesn't for too long (because it is stuck for whatever reason) the whole process is to be killed—not just threads.

It is always safe to kill, but that should only happen when it appears no work is being done for too long. As for the cause, I don't want to make any assumptions. In the worker thread, "anything" can happen. Part of my question is whether another thread (with which there is no interaction) is a safe enough place to monitor the worker and potentially kill the whole process, or that I need stronger isolation. Could something nasty happening in one thread cause issues for the whole process (not crashing but effectively blocking it completely)? I don't know; that's why I'm asking. And if so, is a Windows-owned thread acting in the context of that same single process any safer? Again, not familiar enough with Windows internals.

@Lars Fosdal The issue isn't really the restarting itself, but making sure nothing (except for continued operation) will stop that code from running.

Thank you for the surprisingly large number of responses so far. To clarify: my intention in every case is to let Windows restart the service, so cleaning up after handles and all should not be an issue. What I'm looking for, is the simplest solution that can be depended on to get that done. Using child processes for all the work means quite a big refactoring that I'm willing to do only if it is actually more robust than 1 or 4 (easy to implement), which I'm still not sure of. I don't really need any of the other benefits they provide.

I'm writing a Windows service, the continued operation of which is important. To deal with crashes, the OS can be configured to automatically restart it. What may also happen, however, is it just getting stuck somehow. I've already seen this happen when network calls go wrong; once OpenSSL got into a faulty state and blocked forever. Now I'm looking for the best way to terminate the process when it seems like it's no longer doing any work for too long. A few options I can think of:

1. Doing the work in a separate thread that is monitored from the main thread

2. Doing the work in a child process that is monitored by the parent

3. Doing the work in the parent process that is monitored by a child

4. Doing the work in a process that is scheduled to be killed by Windows

First of all, I'm not sure if I need process isolation or not. Can one thread cause the whole process to hang, i.e. all threads, including those it has no interaction with? Process isolation might be ideal, but it also brings complications (e.g. logging to the same file) that I could do without.

If 1 is not good enough, do I really need to go for 2 or 3, or could 4 work well enough? If a schedule timer-queue timer is fired to terminate the process when stuck, does that running in a Windows-owned thread make it any different (in terms of reliability) from 1? Would it still work when the process is completely stuck? Is there another, better way to let Windows handle the situation?

Between 2 and 3, is there any meaningful difference?

15 hours ago, eivindbakkestuen said:

Just grab NexusDB Embedded Free from GetIt, compiles directly into your windows app, only your exe to distribute.

Proprietary databases are a no go IMHO, when there are plenty of excellent alternatives.

I was surprised by @dummzeuch's benchmark results, so ran my own. In short, I created 44350 strings by taking 8870 English words and appending 1 to 5 to them, added half of them (index odd) to the data structure, to finally check for all of them if they were present. The experiment was repeated 100 times, and the listed results are the average total time for each action:

TStringList
- Add: 143.44 ms
- TryGet: 123.09 ms

TStringHash (Size = 256)

- Add: 23.58 ms

- TryGet: 62.56 ms

TStringHash (Size = 512)

- Add: 17.09 ms

- TryGet: 40.54 ms

TStringHash (Size = 1024)

 - Add: 13.47 ms

- TryGet: 29.3 ms

TStringHash (Size = 2048)

- Add: 11.45 ms

- TryGet: 21.68 ms

TStringHash (Size = 4096)

- Add: 10.2 ms

- TryGet: 18.41 ms
TStringHash (Size = 8192)
- Add: 9.49 ms
- TryGet: 16.28 ms
TStringHash (Size = 16384)
- Add: 9.17 ms
- TryGet: 15.21 ms
TStringHash (Size = 32768)
- Add: 8.73 ms
- TryGet: 14.24 ms
TStringHash (Size = 65536)
- Add: 8.68 ms
- TryGet: 14.03 ms
TDictionary
- Add: 10.2 ms
- TryGet: 16.09 ms

TSynDictionary

- Add: 9.02 ms

- TryGet: 14.99 ms

If you're really interested, I can send you the code, but I think these results make sense. As long as the number of buckets is somewhat appropriate, a TStringHash outperforms TDictionary even. Possibly the former has a more specialized hash function, or simply does less other work (e.g. no accounting for reallocation). Note that I did make it fair by checking for duplicates in TStringHash too. If you don't need that, it becomes even faster.

10 hours ago, dummzeuch said:

Yes:

I thought of improving it too, but it probably would be more work than writing my own. Just the fact that it's based on TStrings adds overhead that I would like to avoid.

 

(I guess you meant THashedStringlist, because if I remember correctly, TStringHash isn't even exported.)

TStringHash is a completely different class than THashedStringList and gives you exactly what you asked for: string to Integer (Pointer). I've been happily using it in D6 and it's still publicly available in D2010 (the latest version I have at hand). Sure it might not be the absolute optimal solution, but it will be a noticeable improvement over the status quo, is very understandable, and doesn't need any third party code.

Did you consider IniFiles.TStringHash? It has a fixed number of buckets, but you could compose it and implement your own reallocation logic if desired. Also, you could inherit from it to use a better hash should the default not be good enough.

I would suggest to at least not go the JSON route. If you don't want to go fully relational, and don't have too much nesting, you could use something like the Entity-Attribute-Value (EAV) pattern. When done correctly, that's still quite workable with a RDBMS like Firebird while already offering greater flexibility (with which also comes greater responsibility). It could strike a nice balance.

If you can get away with its limited functionality (which is sort of the point), just use libsodium.

@Vincent Parrett Later versions of Inno Setup have several options for retrying, so it's not a problem for me. The only thing that still doesn't work to well is aborting the compilation when it's signing.

I have a certificate issued by Comodo and let Inno Setup take care of it. As also covered in the blog post linked above, if you obtain a timestamp from a trusted party (special server) when signing, the signature will remain valid. For me the most useful resource was this: https://www.zabkat.com/blog/code-signing-sha1-armageddon.htm.