Why a non-coding interface to Deep Neural Networks could radically expand access to Machine Learning
Unreal Engine's elegant Blueprint visual programming interface
Unreal Engine's Blueprint feature is a visual scripting tool that allows users to create gameplay mechanics, AI behavior, and other functionality in the Unreal Engine without writing any code. This means that instead of writing lines of code, users can create logic by connecting together visual blocks that represent different actions or functionality.
instead of writing lines of code, users can create logic by connecting together visual blocks that represent different actions or functionality
In terms of deep learning, this means that Blueprint could be used as a graphical programming interface for packages like PyTorch and TensorFlow. This would allow users to create and train deep learning models without having to write any code. Instead, they could use Blueprint to create the necessary connections and logic for the model, using the visual blocks provided by the tool.
For example, a user could use Blueprint to create a deep learning model that recognizes objects in a scene. They could connect together blocks that represent different operations, such as image preprocessing, data loading, and model training, to create a visual representation of the model. This could make it easier for users who are not experienced with programming to work with deep learning packages and create their own models.
Unreal's interface shows just how much work can be done in the Unreal Engine
Users could use Blueprint to create the necessary connections and logic for the model
Aside from being much more approachable, Blueprint has the advantage that the connections one makes in the interface must be correct: you can't connect a float to an int, for example, without a converter bit of code (which Unreal generates for you automatically), or a 3 vector (x,y,z) to a scalar number. This proper, visually apparent connectivity between program elements removes many many sources of error when Blueprint coding in Unreal, and this same reduction in errors could be transferred to Deep Neural Network code. One thing that always gets me is the dimensionality of ingoing and outgoing tensor dimensions; imagine if when you dragged a "noodle" to connect one DNN layer to another, the dimensions were apparent and you couldn't drag one to the other without proper dimensionality? How much time and grief would that simple step solve all on its own? And that's just one of the many areas where visually based coding would be a game changer, especially for those who are not experienced programmers.
The Blueprint interface can trace arbitrarily complex logic
This proper, visually apparent connectivity between program elements removes many many sources of error when Blueprint coding in Unreal, and this same reduction in errors could be transferred to Deep Neural Network code
The main criticism of using Blueprint (as opposed to C++ coding) is that individual blueprints can get quite complex both visually and in terms of screen real-estate. Still, however, the visual guides and pre-compiling checks that the Blueprint interface enforces means those without a lot of programming experience can quickly block out best-practices based code--without ever having to type a line of it! That's a pretty cool vision for the future.
The Blueprint interface can at times become complex to read
Overall, the use of Unreal Engine's Blueprint as a graphical programming interface for deep learning could make it easier for non-programmers to get started with deep learning and experiment with creating their own models.
Let me know what you think of this idea in the comments. Thanks!
John Gibbs, President and CEO of Artimatic