Reinventing the Wheel
The drawbacks of vendorscripts were illustrated in a recent article that described how a couple of test automators had collaborated to add stacks and queues to a vendorscript. The point of the article was to demonstrate what test automators could do when they worked together, but the lesson I drew was that vendorscripts can lead test automators to waste a lot of time and effort reinventing wheels. Stacks and queues are elementary data structures that require little effort to implement in standardized languages. I can sympathize, because I've been in the same situation. I have had to build libraries to support string manipulations, date math, and calculating permutations for various vendorscripts. Some vendorscripts have libraries for such things, but the libraries for standardized languages are much larger and more robust.
Back in the Old Days
Lots of nontesting tools have embedded vendorscripts as well. And it used to be even worse. About a decade ago, John Ousterhout decided to do something about it. He designed Tool Control Language (TCL), made it easy to integrate with C, and released it into the public domain. TCL is now used in some public-domain test tools. Ousterhout makes a distinction between system programming languages (such as Pascal, C, C++, and Java) vs. scripting languages (such as Perl, Python, Rexx, TCL, Visual Basic, and Unix shells). System programming languages are better for building from scratch and optimizing for performance. Scripting languages are better for rapid development and reusing code. They are great for test automation. Some of the test tool vendorscripts pre-date Ousterhout's advances—created before TCL and when Perl was still in its infancy. But that was then. There are plenty of options now.
So with plenty of options, vendorscript should be retired. With one exception, there are no third-party books that you can use. You'll have to rely on books from the vendors, which may or may not be frank about the vendorscript's shortcomings. Courses are also hard to come by, expensive, and may require travel. If you are hiring, it will be hard to find people who are already trained. Test automation is hard enough without us having to flesh out the vendorscripts that come with many tools. Developers wouldn't stand for programming languages with such limitations. We shouldn't either.