Tag: Einstein Vision


Salesforce’s new Einstein innovation: Tracking & Protecting Great White Sharks in Southern California

After reading the title, you must be wondering ‘What’s this?’ What is the connection between tracking sharks and Salesforce? But you read it right. Some of Salesforce’s Einstein team members have collaborated with US Santa Barbara’s Benioff Ocean Initiative and are using AI to analyze drone footage in near real-time to track the movement of the great white sharks close to the shore of Southern California. This AI application will certainly help to keep the beaches safer.  

We have seen some amazing AI use cases in recent times. Big data along with enhanced algorithms have taken AI out from research labs to drive some innovative and real-time applications. To name a few of the recent AI incredible applications are the hurricane forecasting system for the Caribbean, personalized classroom lessons, and best lead prediction for sales reps. 

The recent application to track white sharks will certainly help to protect their quickly diminishing population. The significant increase in the great white shark sightings near the popular surfing areas in Santa Barbara had grabbed the attention of the local community as well as the scientists. Benioff Ocean Initiative Director and UCSB professor Douglas McCauley teamed up with Salesforce Director of Research Michael “MJ” Jones, Salesforce Chief Scientist Richard Socher and San Diego State Associate Professor of Computer Science Xiaobai Liu to find out a new way to identify the presence of sharks in the coastal waters using Salesforce Analytics & AI.

Salesforce R&D team then leveraged AI and train the Einstein Vision algorithm which is being used for visual search, brand detection, image recognition, and product identification. With the help of Einstein Vision, they analyzed the drone video in real-time to identify the shark movements without physically tagging them or touching the water. The project was called ‘Project SharkEye’. 

Also Read: Image Recognition with the New Einstein Vision

Salesforce AI team worked with UCSB researchers and gathered the video by flying drones along a 10-mile stretch of beach. Einstein Vision was then taught to spot white sharks in the mighty blue ocean. However, to get real-time insights, computer vision was required. Training the algorithm was not an easy task. Einstein Vision has to learn to differentiate between shark species, such as a leopard shark and a great white. It had to rule out non-shark objects such as a large floating piece of kelp with a shark-like shape. Also, the algorithm has to learn which is a new shark or the same one. With time, the algorithm got smarter. 

Salesforce also wanted to help the local community who were getting their livelihood from the ocean and the tourists. The shops, the schools, local businesses, lifeguards, etc. Salesforce lately offered Field Service Lightning mobile apps to the field workers such as lifeguards, marine biologists, surf teachers so that they can use the app for onsite job management, real-time collaboration, and access information. The app will involve the local community in the white shark tracking and protecting mission as they can avail of all the information using the Lightning mobile app such as how many sharks were spotted and where. 

Project SharkEye was not only one of the all-time great AI use case, but also it allowed Salesforce to be helpful to the community. This initiative brought technology to a place where it was never been in the past. 

Also Read: Salesforce Introduced Einstein Search to Personalize CRM Search Results with AI

To know more about Einstein Analytics and how it can help your business get more insights, please get in touch with us. 


Source: Salesforce


AI, Salesforce

How Salesforce is using AI across its platform

In an interview during Salesforce World Tour in London this month, the head of the Salesforce Einstein product team discussed how they are using top-notch data science techniques and enable their customers to get best experiences in Salesforce’s SaaS platform.

Since Salesforce has acquired MetaMind and appointed its Founder Richard Socher as the Chief Scientist of the company, the primary objective is to implement intelligent features more into Salesforce’s CRM software. This will provide more access and insights to Salesforce’s customers in the sales, marketing, and customer service domain.

This was the idea behind the Einstein brand which was announced during Dreamforce 2016. Since then, each year Salesforce is introducing innovative groundbreaking features in it’s AI platform. Salesforce’s AI and Machine learning in-charge Marco Casalaine and his team work with all the product teams on each cloud platform within Salesforce – be it Marketing, Sales, Service, Analytics or Commerce, to discover areas where AI could be used to make it better. Salesforce has embraced AI in various platforms as well as in MyEinstein platform, which presents the underlying technology to customers looking for customized deployments.

Customized AI with MyEinstein
MyEinstein represents Salesforce Einstein, which is AI for it’s CRM platform. The objective is to allow customers to develop custom AI models on their Salesforce data to help them predict business outcome and take productive business decisions.

Enterprises can develop smarter, more predictive applications which can leverage their Salesforce data and solve their specific business challenges with MyEinstein. So, without the need for a data scientist, an organization can have AI that is customizable for their business.

Einstein Vision
Now you can harness the power of image recognition to resolve a broad range of specialized use cases by leveraging pre-trained classifiers.

Einstein Object Detection
Train models to identify objects within the images and detect types and numbers of the objects at scale.

Einstein Sentiment
You can now leverage the power of natural language processing to analyze the sentiment of text and classify it as positive, negative, or neutral.

Einstein Intent
Use this fully customizable service to classify the underlying intent in a body of text to understand what customers need across all channels.

Future Roadmap
Customers are expecting a lot from Salesforce AI. AI implementations take longer to roll out than usual features. Currently, Salesforce is working on Einstein prediction builder, which is built on the MyEinstein platform and is aimed at putting predictions right where you are in Salesforce.

Image Recognition with the New Einstein Vision
Einstein Analytics, Salesforce

Image Recognition with the New Einstein Vision

Einstein Vision, a whole new launch of APIs for the process of building the AI-powered applications. No sooner this vision has been initiated that Salesforce is making it easier for you to embed the process of image recognition directly into your apps. In fact, rather than taking hands into entirely building and managing the specialized infrastructure to host the deep learning models, it is always better to simply connect to Einstein Vision’s HTTP/REST API for custom image recognition.

Now, every individual can avail the advantage of pre-trained image classifiers, or rather to put it in simple words, train their custom classifiers, to solve a huge number of image recognition. This is indeed done using cases, empowering all the end users across sales, and even marketing to be smarter and predictive.

Why do we need Einstein Vision?

Einstein Vision is used to discover all your products across all the social media channels. This is done to analyze the observational data in healthcare and life science applications. Accordingly, it will enable visual search in e-Commerce apps.

  • Visual Search – A way to expand your ways, so that your targeted audience can discover your products and thereby increase sales.
  • Brand Detection – A process to monitor all your brands across the channels, to increase your marketing reach and conserve the brand integrity.
  • Product Identification – This is a way you can identify your products and at the same time streamline sales processes and customer service.

The Algorithms behind Einstein Vision

It is true that the Einstein Vision utilizes the advancements made in the genre of deep learning and even an underlying architecture of the neutral networks. It is the way to classify images with absolute accuracy. This as well helps in utilizing multiple hidden layers to extract all the hidden features and assign the apt weights and biases to the feature output. There is always an input layer, where mostly all the images are fed, and even there was a hidden layer, which extracts each feature of the image’s contours, the shapes, and even the corners to predict new images eventually.

How does this vision work?

It is true that the Einstein Vision is a part of a supervised learning technique in which a model is thoroughly trained on the prelabeled training data. This training dataset consists of labeled images and is as well loaded into the Einstein via an API call. Then, a REST API call is made to train the dataset, and hence the output is a trained model ID. Finally, an API call is made to predict the label of a new image that has never been seen before.

Hence, the actual strength of the Einstein Vision is its ability to train custom models to completely to recognize the aspects you care about completely. These are some of the advancements in image recognition that are developing in the new era of customer success and enabling companies to predict the market research in a better way.

CEPTES offers a full range of Salesforce services

CEPTES provides a full range of Salesforce services from consultation to migration to implementation, support, and maintenance. For immediate contact, feel free to email us at contact@ceptes.com. We look forward to hearing from you soon!


  1. metamind.io
  2. trailhead.salesforce.com/en/projects/predictive_vision_apex
  3. github.com/rajdeepd/einstein-go-apis
  4. developer.salesforce.com/blogs/developer-relations/2017/05/using-einstein-vision-within-golang.html
  5. developer.salesforce.com/blogs/developer-relations/2017/04/using-einstein-vision-on-Heroku-or-mobile.html