Why not just use TDateTime(0)?
Well, first you should check and ensure that date parsing is the reason of slowdown in your workflow. There won't be much sense in extreme optimization of this routine if you have only 1 date value in 1Mb file.
Then, avoiding string copy is the must-do when reaching the best performance. In your case for such short and fixed-length numbers there's no necessity in Copy
SetLength(s, 2);
s[1] := src[1];
s[2] := src[2];
IntToStr(s);
s[1] := src[3];
s[2] := src[4];
IntToStr(s);
...
Going deeper,
var s: String;
pSrc, pDest: PChar;
begin
SetLength(s, 2);
pSrc := Pointer(src);
pDest := Pointer(s);
pDest^ := pSrc^;
(pDest+1)^ := (pSrc+1)^;
gives 2280% (!) perf gain over Copy when running in a loop (not an strictly clean test though because I allocated a string once and looped 30k times, but you have 5 2-char parts so reusing still makes sense).
Further, instead of filling string + IntToStr, you could just do
`DatePart := CharToDigit(Str[1])*10 + CharToDigit(Str[2])`
where CharToDigit is *inline* function that does
`Result := Ord(aChar) - Ord('0')`
Of course, some validations should be added but that's the idea.