How do you organize developing new features in big projects?

[Mike Torrettinni 198]

My projects are quite big now and it's getting harder and harder working on new features that require a lot of changes at the beginning - they evolve as development goes on. The problem is all the settings and initialization, and then compiling, running the full project just to test simple changes how they work.

So, when I have clear idea of class structures for new feature, I start new project and develop feature as much as possible on it's own, before integrate it into the project. But this is not always possible, especially for features that rely on existing features.

 

Any recommendations on this topic are appreciated.

 

One of my 'quirks' is that I need to compile and test almost every new method, as soon as is written - this means a lot of compiling and running. Right now I'm just not that sure of my skills to write and write a lot of code before compiling and testing. Perhaps this will be useless topic when I can write complete feature in one go and compile much fewer times.

[Vincent Parrett 750]

There is no standard answer to this, and I guess it also depends on what you are using for version control and it's branching capabilities.

 

We try to follow this strategy https://nvie.com/posts/a-successful-git-branching-model/ but not religiously. 

 

I tend to create a feature branch off of my dev branch. If I find that I need other changes that are not specific to (but needed by) that feature, I'll go back and do them in another branch of off dev, and merge that into my feature branch and dev (assuming it's all good and tested). 

 

The main thing with feature branches is to make sure you are merging your dev branch into it often.. that softens the blow when it comes to merging the feature branch into dev when done. 

 

Long running feature branches are a pain in the proverbial, I often have several on the go at any one time... keeping them up with dev etc is a constant process.. I merge often - lots of small merges are easier than one big one with conflicts to resolve on code you wrote ages ago an can't remember specifics etc.

 

HTH

 

[Darian Miller 361]

Lots of info on: https://trunkbaseddevelopment.com/

[Vincent Parrett 750]

20 minutes ago, Mike Torrettinni said:

One of my 'quirks' is that I need to compile and test almost every new method, as soon as is written

Do you have any sort of Automated build or Continuous Integration - this can ease the pain a lot. When I merge and push (we use Mercurial and Git), my CI server triggers builds on the affected branches, runs the unit tests etc and lets me know how it all went - all unattended and automatically. 

 

<plug>

Take a look at FinalBuilder (not free) and Continua CI (free for 1 concurrent build). Of course I'm biased, since they are my products but there are no better tools when it comes to delphi builds. 

https://www.finalbuilder.com/

</plug>

 

[Vincent Parrett 750]

29 minutes ago, Mike Torrettinni said:

I start new project and develop feature as much as possible on it's own, before integrate it into the project

BTW, there is nothing wrong with this, but I really only do that for code that would have zero dependencies on any other parts of the application - ie utilities etc. pretty much all of my open source projects here https://github.com/VSoftTechnologies were written for a project, but could stand alone. 

For adding features to an application, it's best done in the application. 

 

[Mike Torrettinni 198]

I'm the only developer, so is not the reason, but I guess excuse, why I don't have real source control, branching implemented. Same for automated builds, all manual.

26 minutes ago, Vincent Parrett said:

For adding features to an application, it's best done in the application. 

The feature I started working on now, is html report. And is not just dumping data to html, but also playing around with html to create visually 'acceptable' presentation of data. And I know for that I will have to tweak html and css a lot and generate numerous test reports to get the final acceptable report.

I think this is perfect for standalone feature, but to use real data I need it implemented in the project. So, I guess i will start with base feature as standalone and then finalize on real data when integrated in the project.

[Vincent Parrett 750]

5 minutes ago, Mike Torrettinni said:

I'm the only developer, so is not the reason, but I guess excuse, why I don't have real source control, branching implemented. Same for automated builds, all manual.

If I had a $ for every time I heard this argument, well I'd have more beer money 🍺 

 

Using version control really is a must these days, I don't know how you manage to keep track of things without it. There are plenty of zero cost options :

 

Git - I use this for my open source projects - I use Fork as the ui but often just use the command line,
Mercurial - we use this in house - Install TortoiseHg and you have a great UI
Subversion - not my fav but some people like it

 

Along with version control, a good diff tool is invaluable, we use Beyond Compare (written in delphi, built with FinalBuilder, gotta love inception!) - it's a bargain at USD$60 - I am amazed they don't charge more for it, but on the other hand I have 5 licenses so happy with the price 😉

 

Learning version control is not difficult (if you can drive delphi you have some tech skills for sure!), there are lots of tutorials etc on youtube etc. 

So my advice would be to start there. Once you have that ticking along (a few weeks of practice and discipline at the most) then you can move on to other things like build processes. 

 

Of course getting started with version control, builds etc can seem like a lot of overhead, but once you have it mastered it becomes a productivity boost and a life saver (when did I make that change and why? - be sure to add good commit messages!).  

 

[David Heffernan 2345]

3 hours ago, Mike Torrettinni said:

I don't have real source control,

You should drop everything and fix this. I'm sure we've told you this before. We were right then, we are right now. 

[Mike Torrettinni 198]

5 hours ago, David Heffernan said:

You should drop everything and fix this. I'm sure we've told you this before. We were right then, we are right now. 

Sorry, I don't understand your comment. Probably is my English and the way I phrased the question: the question is about how to start implementing new feature into existing project, where compiling and running it takes a while, but I just want to focus on feature on it's own, and compile and run it to see if it goes into the right direction. Just what your experience is, do you start new feature within the project or do you create new, empty project and implement the new feature, and once the feature is developed then integrate it into the existing project. Or a combination, or some other ideas.

 

Branching, merging and version control of the new feature is not issue, this is all taken care of.

[Dalija Prasnikar 1396]

8 minutes ago, Mike Torrettinni said:

Sorry, I don't understand your comment. Probably is my English and the way I phrased the question: the question is about how to start implementing new feature into existing project, where compiling and running it takes a while, but I just want to focus on feature on it's own, and compile and run it to see if it goes into the right direction. Just what your experience is, do you start new feature within the project or do you create new, empty project and implement the new feature, and once the feature is developed then integrate it into the existing project. Or a combination, or some other ideas.

 

Depending on the feature. If I need to test some general concept and/or feature is not tightly coupled with app I will make separate test project until I figure whether it will work as intended. 

 

If feature is tightly coupled with rest of the application I will create new feature branch in that app repository and start from there, even if compiling/running takes time.

 

8 minutes ago, Mike Torrettinni said:

Branching, merging and version control of the new feature is not issue, this is all taken care of.

But, this is what we are trying to tell you, not having VCS is an issue. Because, new feature workflow highly depends on VCS. 

 

I know it may seem that learning how to use VCS takes valuable time, but you can get that time back in a week (literally).

[David Heffernan 2345]

You only feel the need to develop in such an isolated fashion because you don't use revision control.

 

If you had revision control you'd branch and develop your feature. You'd run your existing tests to prove that you had not damaged existing code.

 

Why do you resist using revision control? It will make you far more productive and effective. 

[Lars Fosdal 1792]

We use frames extensively in one of our UI apps, and I've made a FrameTestBench app that has the required plumbing of the main app, but which only instantiates the frame I want to work on.

It reduces the compile time a lot.

 

But - as everybody else says - using revision control really is essential.

 

Once the frame "graduates" from the test bench and needs to go into the main app for further testing, I don't want to have that happen in the current live branch since I may need to do fixes in that branch before the new frame is ready for release. 

[Mike Torrettinni 198]

12 minutes ago, Dalija Prasnikar said:

But, this is what we are trying to tell you, not having VCS is an issue. Because, new feature workflow highly depends on VCS. 

13 minutes ago, David Heffernan said:

Why do you resist using revision control? It will make you far more productive and effective. 

Oh, yes, I have it, just not in the 'usual' way, more it looks like manual version control, not full blown product like TFS, Git... which I understand could be considered as if I'm missing something really beneficial, but not everything has to be perfect to be useful, right? I mean I use more and more Pred function (even though it was marked as stupid), I find it useful for my needs 🙂

 

 

9 minutes ago, Dalija Prasnikar said:

Depending on the feature. If I need to test some general concept and/or feature is not tightly coupled with app I will make separate test project until I figure whether it will work as intended. 

 

If feature is tightly coupled with rest of the application I will create new feature branch in that app repository and start from there, even if compiling/running takes time.

Thank you, I was trying to get a sense of how I'm approaching new features and if I'm missing something obvious. I'm glad great minds think alike 🙂

 

 

[Mike Torrettinni 198]

13 minutes ago, Lars Fosdal said:

We use frames extensively in one of our UI apps, and I've made a FrameTestBench app that has the required plumbing of the main app, but which only instantiates the frame I want to work on.

It reduces the compile time a lot.

I like this idea!

Yes, this is one of the annoying things I need to always modify what the project does, so it compiles fast, ignores all startup steps that are needed for full project to run, when i just want to test how new form looks like.

 

 

[David Heffernan 2345]

18 minutes ago, Mike Torrettinni said:

great minds think alik

Every effective programmer understands the value of revision control

[Lars Fosdal 1792]

Why revision control...
- Tracking changes - what is the change history of a unit? (Yes, you SHOULD write commit comments)

- Comparing changes between current version with any previous version

- Comparing changes between any previous version and another previous version

- Automatic merging - ensuring that every little change makes into the other branch

- Rolling back to a previous version if your changes turned out to be crap

- Linking code changes to your issue management system such as Jira

All at a click of a button, no manual tinkering, and not requiring multiple team members

 

Taking it one step further - using f.x. Continua CI which is available in a free version limited to one task at a time, every time you commit a change, you can 

- Automatically build of each of the changed branches

- Automatically run unit tests and integration tests

- Automatically do translations, code signing, packaging, deployment to wherever you need to deploy

- Export snapshots to your offsite backup system

 

Yes, you can do manual zip archiving of code revisions - but man, they invented the wheel!

[Dalija Prasnikar 1396]

41 minutes ago, Mike Torrettinni said:

Oh, yes, I have it, just not in the 'usual' way, more it looks like manual version control, not full blown product like TFS, Git... which I understand could be considered as if I'm missing something really beneficial, but not everything has to be perfect to be useful, right?

Manual version control is an oxymoron - there is no such thing as manual version control. It is not about something working perfectly or not, it is about manual version control not working at all.

 

All you have are backups of your code at some point in time. Nothing else. 

[Mike Torrettinni 198]

I just asked admins to move source control posts to the new topic in Tips section. It would be a shame to lose them in this thread.

[David Heffernan 2345]

The revision control comments are on topic. Revision control is a key part of how you develop new features in an existing code base. 

[Mike Torrettinni 198]

As this is quite important topic for me, as it it part of my daily work - new features in existing project, lets recap: I have 3 options for a new feature:

 

1. Implement it right into the existing project

+ advantage: you have access to full project framework

- disadvantage: when you want to test out a little change in the new feature, like position of controls, the whole project needs to be compiled and run. A lot of time wasted just for simple form view.

 

2. Start new empty project and develop feature as much as possible on it's own, before integrating into the existing project

+ advantage: brand new project, fast compile, focused only on new feature

- disadvantage: lack of full project framework, can't test dependencies on the full project

 

3. Have a basic framework project that has all 'plumbing' of the full project as needed, without all the dependencies; integrate into existing project, when ready 

+ advantage: fast compile, have access to some full project dependencies

- disadvantage: preparing such basic framework could be a hassle, as each feature could require different parts of full project - possible tweaks in basic framework needed for each new feature

 

Any other suggestions are absolutely welcome!

[Attila Kovacs 629]

12 hours ago, Mike Torrettinni said:

but to use real data I need it implemented in the project

I don't.

[Mike Torrettinni 198]

Just now, Attila Kovacs said:

I don't.

You are right! I wasn't thinking that way, until Lars suggestion... then I connected the dots.

In some cases is hard to replicate the data from full blown project, to have them ready for new project with just a new feature.

[Attila Kovacs 629]

13 minutes ago, Mike Torrettinni said:

- disadvantage: preparing such basic framework could be a hassle, as each feature could require different parts of full project - possible tweaks in basic framework needed for each new feature

Turn this to an advantage. It will pay off.

[aehimself 396]

2 minutes ago, Attila Kovacs said:

Turn this to an advantage. It will pay off.

Modularity is the key. And the beauty of inheritance makes it really-really easy.

Since I started to chip my code to as small blocks as possible, separate classes of course; I realized that I started to move more and more of these one-class units to my Framework folder, out of an applications folder.

And they simply all work together.

[Attila Kovacs 629]

3 minutes ago, aehimself said:

And they simply all work together.

And you can fire up a new project having everything needed in just seconds. Regardless if for testing/new feature/completely new app etc...