ANN: Better Translation Manager released

[Vandrovnik 214]

1) Thank you for the registry tip, replacing it here is not a problem.

 

2) I have downloaded amTranslationManagerInstall-1.0.7241.65070.exe, but is is exactly the same as amTranslationManagerInstall-1.0.7235.50584.exe (I mean filename differs, but CRC of the two files is the same.). Both installers install 1.0.7234.2824.

 

3) I would prefer not to solve the problem during import, but after, because I can better decide which data to keep, or to copy parts from one row to the other (we have Czech, German, Polish and Hungarien and I need to be sure that all translated terms are OK before I decide, which one to delete).

[Anders Melander 1782]

15 minutes ago, Vandrovnik said:

I have downloaded amTranslationManagerInstall-1.0.7241.65070.exe, but is is exactly the same as amTranslationManagerInstall-1.0.7235.50584.exe (I mean filename differs, but CRC of the two files is the same.). Both installers install 1.0.7234.2824.

Bloody hell! The 7234 release was an early beta.

This has happened before. From time to time my web hosting provider messes up and replaces new files with old files. I can see that the new files are gone and only the old file are left.

 

I guess this means that everybody has been downloading obsolete versions.

 

Please stop everything until I have resolved this problem.

 

21 minutes ago, Vandrovnik said:

(we have Czech, German, Polish and Hungarien and I need to be sure that all translated terms are OK before I decide, which one to delete

So you understand all those languages?

 

Anyway I understand the use case now. I'll see if I can cook up a good UI for it.

[Vandrovnik 214]

Strange things happen from time to time 🙂 Of course I can wait.

 

No, I do not understand these languages 🙂 Fortunatelly, it is usually enough to recognize empty cell, cell translated to hungarien/polish and cell, which should contain hungarien/polish text, but there is german text, because someone messed it up 🙂 (Or there are common mistakes like spaces, brackets, colons, semicolons etc.)

 

If it were possible to copy all data from Translation Memory to clipboard as tab-separated list, I could paste it to Excel, sort and find duplicates there, but much more comfortable it would be inside BTM - I would sort by source language (Czech) and click button to find next duplicate in the column which is used for sorting (Czech). Usually duplicates are mistakes, just in a few cases they are correct (as with Volume = Hlasitost, Volume = Svazek).

[Anders Melander 1782]

5 minutes ago, Vandrovnik said:

I can wait

New version up now: http://melander.dk/download/amTranslationManagerInstall-1.0.7251.4.exe

 

9 minutes ago, Vandrovnik said:

(Or there are common mistakes like spaces, brackets, colons, semicolons etc.)

BTM should normalize those when they are applied.

For example if you have the translation "&Foo:" = "B&ar:" in the TM and you apply it to "(foo)" then the translation automatically becomes "(bar)".

I try to equalize case (upper/lower/sentence/title/etc), accelerators, format strings, surrounds ()[]{}, etc. and terminators .:?... There might be more. I can't remember right now.

 

[Vandrovnik 214]

Download of version 7251 works fine, thank you!

 

The equalization - is it possible to customize it somehow? Upper/lower is problematic between Czech and German (in Czech noons begin with lower case, in German with upper case). I have disabled Apply to similar values, will it disable all these equalization and look for exact mathes only?

[Anders Melander 1782]

2 hours ago, Vandrovnik said:

The equalization - is it possible to customize it somehow?

No. But...

 

2 hours ago, Vandrovnik said:

Upper/lower is problematic between Czech and German (in Czech noons begin with lower case, in German with upper case).

OK. I will have to do something about that.

 

For matches from TM the rule is that if I find an exact match in the TM then I do not equalize case (*). For example if I'm translating "Foo" and I find "Foo"->"bar" then I use "bar" as the translation. If I find "FOO"->"BAR" then I equalize "BAR" to "Bar" and use that.
*) I still equalize endings and accelerators regardless of match exactness since the I consider TM data fidelity/consistency somewhat poor.

 

I currently have the following equalization rules:

• If source has an accelerator then make sure target also has one. If possible use the same hot key.
• If source does not have an accelerator then accelerators are removed from the target.
• If source ends with a colon, semicolon or ellipsis then make sure the target also does so, but only add new ending if the target ends with an alphanumeric character.
• If source does not end with a colon, semicolon or ellipsis but the target does then remove the ending from the target.
• If source is surrounded with ( ), [ ], { }, " ", ' ' or < > then make sure the target also does so.
• If source is UPPER CASE, lower case, Title Case, Sentence case or simply Starts with an uppercase letter followed by a lowercase letter then make sure the target is too.

It's implemented in the MakeAlike function in amLocalization.Utils.pas

 

If I'm understanding you correctly then the only safe case rule between Czech and German is the UPPER CASE and maybe the Title Case rule.

 

So if I'm making equalization configurable at what level should it be configurable. I mean I could make everything optional and all values configurable but it will be a nightmare to implement the UI for it and probably far too complex for the user to manage. I'm also guessing that the desired rules would differ between projects or even within projects, between languages.

 

2 hours ago, Vandrovnik said:

have disabled Apply to similar values, will it disable all these equalization and look for exact mathes only?

No.

The "Apply new translations..." and "Apply to similar values" options controls if new translations are automatically applied to other identical or similar (via normalization) terms.

For example if I translate "Foo" to "Bar" in once place, then it can automatically apply this translation to all other instances of "Foo" in the project. If the "similar" option is enabled then it will also apply the translation to "&Foo"->"&Bar", "FOO"->"BAR", "[foo]"->"[bar]" etc.

[Anders Melander 1782]

1 hour ago, Anders Melander said:

at what level should it be configurable

Here's what I have implemented for now. It's very rudimentary and applies globally:

[Vandrovnik 214]

Thank you for answers and new settings - that is fine, I would just disable Character case for our languages combination.

Even all upper case in Czech may be different in German; Czech "DPH" (English "VAT") is "MwSt" (or even MwSt. with the period?) in German (and I have no idea about Polish and Hungarian, just know that in Hungarian our customer often uses upper case even for names etc.), so it is safer to disable Character case for us.

 

If you add the button for finding duplicates in Translation Memory, please could this button follow the same rules for equalization as the rest of BTM? Because from the past, we have lots of such semi-duplicates: "E-mail" (as a column title) and "E-mail:" (as a label) etc., because with ETM we needed both. Now in BTM I have always to select which to use, so I would like to keep just "E-mail" and let BTM add corect endings as necessary.

 

What is the right/easy way to correct entries in Translation Memory? I use it to translate a form, manually check the result and find a mistake (missing space etc). If I correct it in the form, it will be replaced the next time I run the translation from TM on the form. So I would like to use Add to TM, but I get:

 

If I press Yes, a duplicate record will be created; what I need is to replace original German text with the new one. Or is it possible to go directly to TM to the record which was used to translate this term to German and make changes there?

 

[Anders Melander 1782]

On 11/9/2019 at 10:05 AM, Vandrovnik said:

If you add the button for finding duplicates in Translation Memory, please could this button follow the same rules for equalization as the rest of BTM?

Done. The action performed on values to check for equality isn't called equalization though. It's called normalization. Internally, in the source, I call it Sanitation (i.e. I remove all the junk).

 

Anyway, here's what I have implemented:

 

The "duplicate view" is invoked via the popup menu:

 

When viewing duplicates a new column is added. This column displays the sanitized value of the selected language. You can control what sanitation rules to apply.

Only rows that have duplicate values are displayed in duplicate view.

 

In case you wonder why I'm not grouping on the Duplicate column; I tried it but found that it just cluttered the user interface and made the grid too cumbersome to navigate.

 

I'll try to get a new version with these changes released tomorrow or Tuesday.

 

On 11/9/2019 at 10:05 AM, Vandrovnik said:

If I press Yes, a duplicate record will be created; what I need is to replace original German text with the new one. Or is it possible to go directly to TM to the record which was used to translate this term to German and make changes there?

I understand the problem.

The dialog is already trying to communicate too much information and has become too complex for a standard message dialog. I have already redesigned it a bit (see below) but in order to add a "Replace" action I would have to switch to using a full blown Task Dialog or even better a custom dialog. That however will have to wait.

 

I think the best I can do in the short term is to add a "Find in TM" action. This would open the TM, try to locate the currently focused Source/Target value pair and focus the target value in the TM.

[Vandrovnik 214]

Looks very good!

 

Locating the record in TM is fine.

 

Looking forward to new version 🙂

[Anders Melander 1782]

@Vandrovnik A new version with the Duplicate View is now available:

 

http://melander.dk/download/amTranslationManagerInstall-1.0.7254.4932.exe

 

I had a bit of trouble getting this version up as BitDefender kept insisting that amResourceModuleBuilder.exe contained the Gen:Variant.DCry.1 ransomware.

In the end I had to simply include the previous version of that file instead.

 

Description of the Duplicate View above.

 

I have also included the "Locate in TM action:

As the hint says, the action attempts to locate the focused source/target pair. It tries to be a bit intelligent about it in case there are multiple candidates (due to sanitation/equalization) so it will give priority to exact matches, case insensitive matches, sanitized matches and equalized matches, in that order. If a match is found the TM is opened and the matching row is selected:

 

This version also includes Copy to clipboard (as CSV) and Save to CSV. Next up is Paste and Import from CSV.

[Vandrovnik 214]

Excellent!! Thank you so much!

 

I hope now I will be able to get rid of Embarcadero's TM - yeasterday it took almost two hours to prepare new version of app, which seems to take just a few second in BTM.

[Anders Melander 1782]

New version released: v1.0.7271.54299
http://melander.dk/download/amTranslationManagerInstall-1.0.7271.54299.exe

 

Changes since v1.0.7254.4932

New features:

• GNU GetText PO file import.

Improvements:

• Copy to clipboard not uses tab delimited CSV for better integration with Excel.
• Normalization rules can now be configured.
  
• New validation rules: pipe | and surround ()[]{}<> mismatch.
• Validation warnings can now be dismissed and resolved.
   
• New equalization rules: Leading space and Trailing space.
• Duplicates in Translation Memory lookup results are now ranked by the similarity of the translation source to the value being looked up.

Other:

• Installation of the command line tool is now optional.
  Primarily because it is currently being flagged as a false positive by several virus checkers.
• Removed dependency on midas.dll
• A slew of performance and usability improvements and bug fixes.

 

Thank you to all those that have sent me suggestions, feedback and bug reports.

[Girish Patil 3]

Anders,

Sorry for the late post on this... Thank you very much for this really useful tool.

I remember adding support for your GIF units in our report export products about two decades ago.

 

Thanks,

Girish

www.gnostice.com

Edited December 23, 2019 by Girish Patil

[PulsarNova 1]

Excellent job, excellent application ! 🙏

 

Original ITE, ETM was really very disappointing sometimes (being sarcastic here 😑).

I think Embarcadero (currently owns the RAD Studio) should hire you for this excellent work, you are really covering up their a.. 

 

 

Like me, very old fans of Borland's Delphi and C++Builder (my favorite) do still like and write codes with this wonderful tool but sometimes it disappoints us in the very simple-stupid corners of programming life.  

 

And you, my friend today you are my number one savour of the day.

 

Thanks a lot for your great work.

 

Sincerely, 

Edited May 24, 2020 by PulsarNova

[Jan Doggen 0]

Sorry, I can't get this to work

 

- New Project based on the Helloworld example
- Source language was set to EN/US, target to Dutch
- Update project to read in the forms/resources

 

If I click Build/Dutch or Build/all languages,  in TResourceModuleWriter.BeginWrite, the line
FResourceHandle := BeginUpdateResource(PChar(FFilename), True);
gives an OS error 193 %1 is not a valid Win32 application.

FFilename is D:\BTM\Examples\HelloWorld\Win32\Debug\HelloWorld.NLD at that point,
so indeed, not an application 😉 What can be going on?

 

 

[Anders Melander 1782]

1 hour ago, Jan Doggen said:

I can't get this to work

I'm assuming you have compiled amTranslationManager.exe yourself since you seem to be running it in the debugger.

• What source code revision are you building from (git hash or timestamp)?
• What version of Delphi are you using?
• What version of Windows are you using?

The helloworld.nld filename is correct. The file is supposed to be a DLL that only contain the translated resources (i.e. no code). The NLD file type is just the ISO 639-2 language code for Dutch (Netherlands).

[Jan Doggen 0]

On 7/26/2020 at 4:02 PM, Anders Melander said:

Thanks for the quick reply, Anders.

Indeed, I downloaded anders_melander-better-translation-manager-a9e47ac90e7f.zip from https://bitbucket.org/anders_melander/better-translation-manager/src/master/, and compiled it in Delphi 10.3 (some compiler warnings about comparing Chars with #$1000 large number, I replaced that with Ord(Char)= $1000)

Running on Win10.

(How) Is Windows supposed to know that a .NLD is a dll?

Quote

 

I'm assuming you have compiled amTranslationManager.exe yourself since you seem to be running it in the debugger.

• What source code revision are you building from (git hash or timestamp)?
• What version of Delphi are you using?
• What version of Windows are you using?

The helloworld.nld filename is correct. The file is supposed to be a DLL that only contain the translated resources (i.e. no code). The NLD file type is just the ISO 639-2 language code for Dutch (Netherlands).

 

Edited July 27, 2020 by Jan Doggen

[Anders Melander 1782]

1 hour ago, Jan Doggen said:

(some compiler warnings about comparing Chars with #$1000 large number, I replaced that with Ord(Char)= $1000)

The revision you used should be Okay.

I'm curious about where you got that warning. There should be no hints or warnings when compiling and no where in the source is there a #$1000.

Anyhow, if you're up for it you can try downloading the precompiled application instead and see if you can reproduce the problem with that:

http://melander.dk/download/amTranslationManagerInstall-1.1.7465.55536.exe

 

I committed a change yesterday that replaces the empty DLL stub I use to generate the resource modules with a smaller file. I haven't tested the new or the previous version of the DLL stub on Windows 10 though, so I will do that right now.

 

1 hour ago, Jan Doggen said:

(How) Is Windows supposed to know that a .NLD is a dll?

Windows doesn't use the file type for anything. It's the file header (the PE header) that identifies the file as a DLL. The "DLL" file type is just a convention - and the default file type for DLLs. You can call them anything. For example Control Panel applets (*.cpl) and Screen Savers (*.scr) are also just DLLs.

 

The resource modules are loaded by the Delphi RTL using the LoadLibraryEx API function. See LoadResourceModule() in the system.pas unit.

 

When your application starts the RTL by default looks for a file with the same name as the application and an extension that matches the current locale. If that file is found it is loaded with LoadResourceModule.

See GetResourceModuleName() for the (broken) algorithm that calculates the name of the resource module file. It's pretty obfuscated due to all the conditional defines.

[Anders Melander 1782]

1 hour ago, Anders Melander said:

I haven't tested the new or the previous version of the DLL stub on Windows 10 though, so I will do that right now.

Works fine for me on Window 10.

[ConstantGardener 31]

Hello Anders,

 

thank's for this great tool! Work's great for me in normal Forms and Frames, but with Ressourcestrings it doesnt.

 

Here is a screenshot, the Name of the resourcestring and the content of the string are not right! Any Idea?

 

Edited July 28, 2020 by ConstantGardener

[Anders Melander 1782]

3 hours ago, ConstantGardener said:

Here is a screenshot, the Name of the resourcestring and the content of the string are not right! Any Idea?

Thanks.

My guess is that the DRC-file was out of sync with the EXE file at the time you created the project.

You should compile the application again and make sure you generate the drc file (see below). Then you can Update the translation project and the resourcestring names and values should now match.

 

Unfortunately the existing resourcestring translations will be marked obsolete since the source name/value pairs have all changed.  The only way to recover the resourcestrings you have already translated is to add the translated resourcestring values to the Translation Memory before the update and then use the Translation Memory to apply the translations again after the update.

 

The DRC-file contains the mapping between resourcestring names and string resource IDs. It is created by the Delphi compiler if the "Output resourcestring .drc file" linker option is enabled. Since this isn't obvious I have just updated the DRC-file prompt to state this fact:

 

I have also added a check to display a warning if the timestamp of the EXE and DRC-files are too far apart (>10 seconds), indicating that they are probably out of sync:

[ConstantGardener 31]

Hello Anders,

this work's now for me! Thank You!

[Jan Doggen 0]

21 hours ago, Anders Melander said:

I'm curious about where you got that warning. There should be no hints or warnings when compiling and no where in the source is there a #$1000.

It's not $1000 specifically 😉

 

unit amLocalization.Import.XLIFF;

 

function IsAnsi(c: char): boolean; Inline;
begin
  // Result := (c <= #$FF); [JD]
  Result := (Ord(c) <= $FF);
end;

 

unit amLocalization.Persistence;

 

//    case Value of [JD]
//      #$20..#$7E,
//      #$A0..#$D7FF,
//      #$E000..#$FFFD:
    case Ord(Value) of
      $20..$7E,
      $A0..$D7FF,
      $E000..$FFFD:

 

unit amLocalization.Normalization;

 

function IsAnsi(c: char): boolean; Inline;
begin
  // Result := (c <= #$FF); [JD]
  Result := (Ord(c) <= $FF);
end;

 

21 hours ago, Anders Melander said:

Anyhow, if you're up for it you can try downloading the precompiled application instead and see if you can reproduce the problem with that:

http://melander.dk/download/amTranslationManagerInstall-1.1.7465.55536.exe

 

Ouch, that version got an error immediately on startup. That is the Invalid source language ID: 2000 that I also get when running from the debugger (I put ignore/handle breakpoints around those when testing). I have attached the bugreport.txt. That error occurs a second time after choosing the Helloworld example.

I could then continue the program.

After setting target language to Dutch, choosing Update, then Build (all languages) worked.

I assume that if I download new sources these will work too, but I'll wait until the constant warnings are out as well. I have no need to run from the IDE, that was just out of curiosity.

 

 

bugreport.txt

[Jan Doggen 0]

New question: I just reread the documentation and it says that "if you run the application where the regional settings have been configured as "German (Germany)", then the HelloWorld.DEU resource module will be automatically loaded and all translated texts will appear in German." Can a program be written in such a way that it picks a *specified* language file. Preferably at runtime (sometime during program startup). Or is BTM just not the tool for that?