Salesforce released TransmogrifAI, a machine learning library written in Scala that runs on top of Spark. This can be potentially deployed on any cloud such as Heroku/PostgreSQL platform. What all is involved in TransmogrifAI?

• Language: Scala
• Underlying engine: Apache Spark data processing engine
• Deployment platform: A standalone local machine or cloud platform like Heroku

Let us explore a bit more about these new players in the scene and whether they will align with our need to build robust machine learning models. The entry barrier to using the TransmogrifAI library is likely to be the new tech stack that a typical Salesforce developer needs to scale up to.

Learning Scala pays well

It is tempting to just start or re-start your career with Scala because it is a well-paying one. See a snapshot below of languages associated with top salary receiving participants in a survey done by Stackoverflow.

Spark is the framework of choice

Spark is one of the top 5 frameworks in which respondents wanted to continue working. The following snapshot is again from the same Stackoverflow survey.

PostgreSQL is a loved Database

Second, the list is PostgreSQL that is supported in cloud platforms such as Heroku. The following snapshot is from the same StackOverflow survey.

Heroku platform is popular

Though not in the Top 5 list, Heroku was the 12^th popular platform amongst respondents. The following snapshot is from the same StackOverflow survey.

Making a case for Scala

If you are a fan of Python language, you may be slightly disappointed that the Salesforce team chose Scala as the language for the new machine learning framework called TransmogrifAI.
Why Scala?

• Unlike Python, Scala is a compiled language. Scala source compiles to Java bytecode so that the resulting executable code runs on a Java virtual machine.
• Code written in it gets executed much faster (comparing to pure Python)
• Apache Spark, the data analytics engine, is built in Scala language

Starting with Apache Spark

Apache Hadoop is the open-source implementation of the MapReduce. Apache Spark is an enhancement to Apache Hadoop for distributed processing of large datasets. Spark performs better than Hadoop in handling in memory computations. Spark introduces a data structure called Resilient Distributed Dataset that enables better in-memory computations. Spark internally uses Apache Hadoop Yarn for cluster management.

Distributed machine learning

Here are a couple of reasons why we require distributed machine learning i.e., code that runs over not just a single machine but across multiple machines:

1. Ability to handle real-time data: Say, we are talking about a self-driving car. A lot of sensor data is going to come in and the on-board computer has to process them and provide real-time direction to stop the car if it spots a child crossing the road.
2. ML activities that need to be completed fast: For example, How soon can we complete the training process? With distributed computing and RDDs, we achieve it faster.

Need for Spark when the machine learning use case is simple

Spark can take advantage of multiple processor cores in a single machine too. i.e., It is possible to set up a Spark cluster in a standalone machine. In such a case, Spark will take advantage of this multi-core single-node machine like how it will work over a cluster of machines.

Not just another library for Machine Learning

There are quite a few machine libraries that exist already in the market such as Apache Spark MLib. Salesforce TransmogrifAI now makes it easier to use Salesforce data(types) more practically in machine learning. Salesforce team claims that a lot of simplification and abstraction is done not just for pre-processing of data and dynamic selection of models, but all through the programming approach as well.

Need help with Salesforce Machine Learning?

Call us at 855-Mirketa or write to us at info (at) mirketa.com to get a FREE consultation on how to get started with Salesforce Machine Learning.