Wanted to give a little update on the situation in case it could help someone else in the future. For several reasons as a father of 6, I couldn't quite find the time to deep dive into the issue since I actually was able to use the van even though the issue persisted when engine was cold. However, as of maybe a month ago we started having the faultcode "P00AF00 - Turbocharger/Supercharger Boost Control "A" Module - Performance" show up even when the engine was at running temperature.
Just like before, it was mainly (and maybe even only) on the 4th gear it got triggered - this made me really confused since I couldn't really understand why that would matter. After the error occurred about 10 times, sometimes putting us in quite a vulnerable position, it was time to swap out the actuator that I managed to get a replacement unit for by the seller I bought the turbo from...
Started by just unscrewing the current actuator and plugging in the new one to compare the voltages when both were at ambient air-pressure - low and behold, the new actuator gave an even higher voltage meaning it would probably have caused even more of an issue. I think I sourced the entire web for the OEM actuator and finally found the partnr 03N198716N witch is a marked as a repair kit. You can clearly see on the image that it is adjustable and this is of course to be able to set it in the right position for the ECU to interpret the signal correctly.
Since I didn't fancy spending another 200 euro and the fact that I already had another replacement unit, I had to resort to some DIY and here is my solution - though not in any way pretty. I cut the actuator and threaded both parts and finally inserted a extension nut no once merge the two parts again, but this time with the ability to adjust the length.

When the new actuator was fitted and adjusted to 0,747v @ -700 mbar, the adaptation of the turbocharger completed on the first go and the van ran like a dream. Not even a glimpse of hesitation on 4th gear. With the knowledge of this, I think the actuator on the re-manufactured turbo is indeed a OEM-part, just that it only worked on the car it had been installed and configured to from the factory since the voltages were different between the both actuators I had in my hands. Thus VW manufactures a repairkit for the parts needed - that is adjustable.
Regarding the issue only appearing on 4th gear, I have a thesis that the ECU requested a specific narrower value on 4th gear that the actuator couldn't accurately deliver, whereas on the other gears it specified a wider range. I did notice whilst troubleshooting the latest incidents, that the actual value on 4th gear was about 30% instead of the usual 16% on the other gears on heavy load.
It may forever remain a mystery, but the main thing is that it finally works and albeit it took quite some time to figure out - I finally feel like I have a reliable van again!