It would make sense only if your data consist in text files, and you want to keep versioning of the information.
A regular SQL database would replace the old data, so you would need to log the old values in a dedicated table.
You can use the git command-line for all features you need.
Just call it with the proper argument from your Delphi application.
But I would rather take a look at https://www.fossil-scm.org/home/doc/trunk/www/index.wiki
It is an efficient Distributed Version Control Management system, very similar to git, but with a big difference:
"Self-Contained - Fossil is a single self-contained stand-alone executable. To install, simply download a precompiled binary for Linux, Mac, or Windows and put it on your $PATH. Easy-to-compile source code is also available"
So you could be able to much easier integrate it to your own software.
It has some other nice features - like an integrated Web Server - which could be easy for you.
Here also, you would need to call the fossil command line from your Delphi application.