What everyhing else do you put under source control, apart from Delphi projects?

[Mike Torrettinni 198]

I just had a little issue with a website backup and hard time rolling back some changes, so decided to put it under the source control.

I wish I had done this sooner, so I'm thinking what else makes sense to put under source control, like: documentation, various scripts.

 

Do you also put 3rd party source code under source control? I rarely change anything, but I do have some patches I apply to some components on new install.

 

What do you guys have under source control?

[Dalija Prasnikar 1396]

14 minutes ago, Mike Torrettinni said:

Do you also put 3rd party source code under source control? I rarely change anything, but I do have some patches I apply to some components on new install.

Yes. Usually in separate repository, but sometimes in the same as the project which is using them.

 

14 minutes ago, Mike Torrettinni said:

What do you guys have under source control?

Anything I can lay my hands on... my book manuscripts, documentation, illustrations and images, 3rd party source code, my own source code or all kinds, recipes, to do lists, basically all kinds of documents that can (or cannot) change with time.

[dummzeuch 1505]

For a while I put Word/LibreOfficeWriter and Excel/LibreOfficeCalc documents in subversion, but since they are binary it's pretty pointless, because one cannot easily find what changed.

 

I put some configuration files (linux) there and of course source code of all kinds, including third party sources (and use these via svn:external).

Edited September 9, 2021 by dummzeuch

[Pawel Piotrowski 18]

Usually I put as much as possible in svn.

 

We have even some projects where we put the executables and dll files and all the files needed for a deployment into the svn.

 

That is great because a tester can just update his working copy and test the newest build. Or he can go back and tell us when a bug was introduced.

For the team it is also great. No need to sync images, dlls, configs or other data through some other tool. Just Update your working copy and you are good to continue your development.

 

If you are worrying about the repository size on the server, remember that SVN stores delta files even for binary files. Maybe not as gracefully as for real text files, but disk space is much cheaper then developer time.

[David Heffernan 2345]

Everything that you create by hand (as opposed to being created by a script) goes under revision control

[Lars Fosdal 1792]

Code, scripts, configurations, schemas, graphics, etc in git

Any kind of Office 365 Docs in Teams/SharePoint (where they also are versioned)

System documentation in Confluence and Ardoq.

Executables are archived and placed in staging folders on the file server by the build server.

 

[mvanrijnen 123]

Database DDL's

 

[Mike Torrettinni 198]

Wow, you guys are really doing it the right way. I guess some learned the lesson, like I just did - the basic backups don't always solve all problem or make it easy to restore older version.

I will need to rethink this whole thing. I have github as backup and I see they have 2GB and 50Gb packages, so it should handle all my stuff.

[Lars Fosdal 1792]

I also use Google Drive as a scratch pad for various purposes.  Autosave and versioning rules!

[Mike Torrettinni 198]

Just now, Lars Fosdal said:

I also use Google Drive as a scratch pad for various purposes.  Autosave and versioning rules!

For files, right? Or are you talking about notes like Google Sheet and Google Docs?

[Lars Fosdal 1792]

Both.  I have scratchpad documents for work-in-progress designs and other notes. 

 

My Greenshot (screenshot clipper) also automatically copies screenshots there - so that I can make any number of screenshots in a row without having to think about storing them as I "shoot". 

It does mean that I have to go in and clear that folder from time to time 😄

 

So, why not a just the local copy?  Well, sometimes I need to access this while on my phone (a question arises, an idea pops up, etc when I'm not near my work computer).

[Mike Torrettinni 198]

4 hours ago, Pawel Piotrowski said:

If you are worrying about the repository size on the server, remember that SVN stores delta files even for binary files. Maybe not as gracefully as for real text files, but disk space is much cheaper then developer time.

I assume same for Git, right? So I can put a lot of non-source stuff (icons, docs, scripts...) into single repository  and it will push only deltas to github.

[Dalija Prasnikar 1396]

12 minutes ago, Mike Torrettinni said:

I assume same for Git, right? So I can put a lot of non-source stuff (icons, docs, scripts...) into single repository  and it will push only deltas to github.

Not really. 

https://jlericson.com/2021/08/24/git_rewrite_1.html 

https://jlericson.com/2021/09/06/git_rewrite_2.html

Edited September 9, 2021 by Dalija Prasnikar

[Lars Fosdal 1792]

There are workarounds for large files and binaries.

https://www.perforce.com/blog/vcs/how-git-lfs-works

[Darian Miller 361]

I would think a little bigger.  Here's an easy thought process.   Set your computer on fire.   (If you are on a team in an office - consider a natural disaster destroyed your office location.)   Now, get all new equipment and see how painful it is to get back up and running without having to recreate any previous effort.  Can you do that today?

 

There are multiple ways to minimize this pain.  The most common way to protect part of your investment is to use version control on your source code.  An additional tool could be snapshotting your VM and keeping the backups offsite.  Keep in mind the 3-2-1 (or 3-2-3) backup strategy.

 

Ideally, you would hit a few buttons to rebuild your systems and not even lose your current edit session.  But the amount of effort and expense that you are willing to put in pretty much dictates the amount of effort it will take to rebuild.

 

 

[dummzeuch 1505]

11 minutes ago, Darian Miller said:

I would think a little bigger.  Here's an easy thought process.   Set your computer on fire.   (If you are on a team in an office - consider a natural disaster destroyed your office location.)   Now, get all new equipment and see how painful it is to get back up and running without having to recreate any previous effort.  Can you do that today?

For me the biggest headache in that scenario is e-mail. It's stored on a server and in a backup, but since nobody but me apparently ever bothers to delete or at least archive old stuff, every server crash that kills the email means restoring many gigabytes of email before new emails can be received.

 

On my work PC, all I really need is the OS + the email client + Delphi and TortoiseSVN. The most work would be to install all necessary components in the IDE. That's mostly automated though, but there are always glitches.

 

Everything else is on the server (and backed up) or is test data that can be easily replaced (usually data from the server copied locally for performance reasons).

[Darian Miller 361]

3 minutes ago, dummzeuch said:

means restoring many gigabytes of email before new emails can be received.

That's a pretty common issue.  Once you get huge amounts of data in one spot, the only solution is snapshots, if the source supports it.  We had a 7TB database file and trying to make a full copy of it once/night took forever just to go from point A to point B.  We relied on snapshots but liked to always have one full copy as well.

 

There are solutions out there for an optimized backup up Exchange server but I haven't used them.  We just prayed on occasion that the email server didn't crash as it would take a number of hours to get back up and running.

 

[Mike Torrettinni 198]

4 hours ago, Dalija Prasnikar said:

Not really. 

https://jlericson.com/2021/08/24/git_rewrite_1.html 

https://jlericson.com/2021/09/06/git_rewrite_2.html

 

3 hours ago, Lars Fosdal said:

There are workarounds for large files and binaries.

https://www.perforce.com/blog/vcs/how-git-lfs-works

 

Thanks, interesting. Definitely something to think about, but for sure need to clean up any large files, like zip, pdf, psd... also found a few 150MB .rsm files.

[Mike Torrettinni 198]

3 hours ago, Darian Miller said:

Keep in mind the 3-2-1 (or 3-2-3) backup strategy.

It says: "Note that you need to ask your backup vendor specifically whether they offer a 3-1-2 as shown or if they offer only a 3-1-1 (one copy in the cloud.)  Don’t accept only a single copy in the cloud – it simply doesn’t offer the protection afforded by modern continuity and backup solutions."

 

Is anybody really doing this? Requiring their backup vendor to have 3-1-1/3-1-2 backup of their data?

So, if I have have 3 backup copies of my source, and vendor has 3 backup copies of what I upload to cloud backup... there are 6 backup copies of my source! And if that backup vendor is using a 3rd party backup vendor for one of the copies... 🙂 🙂 🙂

Edited September 9, 2021 by Mike Torrettinni

[David Schwartz 426]

19 hours ago, Mike Torrettinni said:

 

What do you guys have under source control?

I back-up anything I can't get from somewhere else or recreate quickly.

 

One thing most people forget about is ... Windows Registry settings!

Another is INI files that aren't kept with your source or EXEs.

 

If you're working inside of a VM, do you mirror your backups somewhere? I don't like backing up my VMs because they're so damn big and they don't seem to get delta'd very well.

 

If you use something like git, the master could be on the VM or the host or in the cloud somewhere. But a lot of folks forget about that and just keep the files local.

 

You should keep a copy of everything both inside and outside of the working environment!

[Mike Torrettinni 198]

3 hours ago, David Schwartz said:

One thing most people forget about is ... Windows Registry settings!

You backup the whole Registry or just patches you run manually after installations?

[David Schwartz 426]

26 minutes ago, Mike Torrettinni said:

You backup the whole Registry or just patches you run manually after installations?

As I said, I back-up anything I can't get from somewhere else or recreate quickly.

 

But you're making an assumption here, which is exactly why I said most people forget about this. (Hint: what were YOU referring to regarding saving/backups? And what do you think I was referring to?)

[Mike Torrettinni 198]

Just now, David Schwartz said:

As I said, I back-up anything I can't get from somewhere else or recreate quickly.

 

But you're making an assumption here, which is exactly why I said most people forget about this. (Hint: what were YOU referring to regarding saving/backups? And what do you think I was referring to?)

I've never done any restore that involve registry settings, except running registry patches like regarding disabling some Delphi packages that slow down IDE. So, I was just trying to understand what you are referring to, perhaps something it's useful for me too, but never realized that's a good idea to have backup of.

[David Schwartz 426]

The Registry is a persistent store that is available for any Windows app to use. It replaced the uncontrolled and often abused use of INI files.

 

Maybe your apps don't use either one, which is fine, but it's fairly common to use one or the other to do things like saving the states of different forms and where the users are in some work process. It's simple data and not usually worth putting into a DB if you're not already using a DB.

 

INI files are just files with a .INI extension, so they're easy to capture along with other files. But Registry entries are another thing entirely. They can be simple or complicated. 

 

(The last place I worked, we were forbidden from using the Registry for security reasons, because it had been so abused by previous devs saving too much PII in it that posed potential HIPAA violations. The Registry tends to be a write-mostly memory bank and when you delete apps, the registry is often left un-touched.)

 

The problems arise when you use it to save login credentials, access criteria, license keys, names, addresses, phone numbers, and things that you don't generally want anybody to see, but may be essential to the operation of your app. The average user is clueless where stuff like this might be saved, or if it even exists, so it's a safe place to hide stuff from everybody but developers. We're "smart rats" and know where to look!

 

Apple has something similar to the Windows Registry, but it's not as obvious. Kind as they are about most things, When you upgrade MacOS, it conveniently "loses" all of the registration keys for Apple's software. I suspect that's intentional on their part.

 

But as a developer, it's smart to stash this stuff away periodically if it contains any hard-to-get-at data. 

 

A lot of user-customization data is saved in the Registry -- eg., it's where Delphi saves all of the settings under Tools -> Options. There's a LOT of stuff there! 

 

That's why, when you update to a new version of Delphi, it's so damned difficult to restore your menus and toolbar to the previous settings! They don't always map nicely from one version to the next.

 

(Personally, I would NEVER subject my customers / users to this level of abuse when updating my software! I want it to be as painless and forgiving as possible.)

 

But the old saw, "out of sight, out of mind" applies to the Registry if you use it for saving anything. Most folks don't think of it when backing stuff up.