As part of the next phase of Syndic8 expansion, I need to be able to deploy code and data files across several machines each time I update it. On my main machine, I write code in one directory, and then use a PHP script called “deploy” to copy the changed files to the location from which they will run.
Over a week ago I started to extend this to handle multiple machines. I was planning to use PHP’s SSH2 functions to implement machine-to-machine file copies. After installing the proper prerequisite libraries, I ran some tests and copied some small files to my target machine. So far, so good.
I then extended my deploy script to use these functions to copy files to the remote targets. It worked fine for the first several files, but then it started hanging. Some investigation and experimentation revealed that this started to happen with files around 100K in length. Believe it or not, I do have individual PHP source files larger than this, some as large as 389K (nearly 17 KLOC).
After a lot of C-level debugging, I cannot figure out what is going wrong here and I will probably end up just invoking “scp” commands as sub-processes. That’s not nearly so elegant, but it will work, and it will let me scale up to more machines, which was the real goal when I started this.