YAHOO IS is to offer its CaffeOnSpark deep learning software to the community. The system was built to enhance Flickr's image recognition engine, and is now available to open source for further development adaptation and general larking about with.
The team explained in a blog post: "We believe that deep learning should be conducted in the same cluster along with existing data processing pipelines to support feature engineering and traditional (non-deep) machine learning. We created CaffeOnSpark to allow deep learning training and testing to be embedded into Spark applications."
The software is available on Yahoo's GitHub repository and can be tested on Amazon Web Services' EC2 cloud or on private Spark clusters.
The team added: "Our goal is to make CaffeOnSpark widely available to deep learning scientists and researchers, and we welcome contributions from the community to make that happen."
The lengthy blog post explained the concept in spectacular detail, but the short version is that it runs on Apache and complements the existing Spark MLib non-deep-learning algorithm that comes as standard.
Yahoo isn't exactly a front-runner in the field but there is little question that the sophistication of the Flickr algorithm bodes well for anyone looking to take the plunge.
Deep learning is becoming a next-big-thing battleground for many of the major tech players. Google has already open sourced its DreamDeep yielding Tensorflow engine with sometimes terrifying results.
Apple recently poached Nvidia's deep learning director shortly after buying deep learning specialist Perceptio, and Nvidia has turned its attention to GPU-accelerated in-car deep learning.
IBM's stalwart Watson supercomputer also got an injection of deep learning last year when the company acquired AlchemyAPI, and the firm has even added a 'personality' courtesy of Cognea.
But don't be fooled into confusing deep learning with intelligence. CERN professor and Blue Yonder founder Michael Feindt told us in an interview last year: "We are very far away from a machine that is 'intelligent' in the very wide sense we attribute it to normally.
"Even the best algorithms do not 'think', 'feel' or 'live', they have no 'self-consciousness' and no 'free will'. This is, and will stay, pure science fiction for a while." µ
Next-gen devices enabled by integrating novel materials on silicon
Plus there's a new way to read comics in town
Find out which six games have most impressed us so far this year
Video shows off upcoming handset in Rose Gold compared to iPhone 6S predecessor