·Jason Yuan
Reallysec Helps Adobe Build an AI-Driven Image Search and Recognition System
Joint work with Adobe across image generation and natural language processing: retrieval, tagging and asset management wired into model capability.
Adobe's Core Products and Cloud Service Ecosystem
The globally renowned software giant Adobe is famous for its flagship products Photoshop, Illustrator, and Acrobat. These core applications are integrated into Adobe's cloud service ecosystem, including:
- Creative Cloud
- Document Cloud
- Experience Cloud
Among these cloud platforms, products where image search is particularly critical (such as Adobe Stock) extensively employ Elasticsearch technology behind the scenes to support high-performance content retrieval capabilities.
Elasticsearch offers very high resilience and stability... and it strikes a good balance between open-source characteristics and rigorous review mechanisms.
Adobe + Elasticsearch Search Ecosystem Landscape
Adobe's self-managed Elastic Stack architecture currently includes:
- 18 production clusters
- Hosting over 10 billion documents
- Real-time write throughput of approximately 6,000 documents per second
Taking Adobe Lightroom as an example, to better support the image search experience, Adobe migrated its system from Amazon Elasticsearch Service to a self-managed Elasticsearch cluster, migrating nearly 3.5 billion documents.
Adobe's content types are predominantly unstructured data, including images, videos, Photoshop files, and more, along with a large volume of structured documents, particularly in PDF format. In this context, the Elastic Stack combined with Adobe's proprietary Elasticsearch plugins effectively supports the following diverse content search scenarios:
- Intelligent search based on computer vision and metadata
- Deep text and mixed content search
- Video and rich media format search
- Enterprise-grade document retrieval
- Content discovery and personalized recommendations
Machine Learning Applications Combining Adobe Sensei and Elasticsearch
Adobe Sensei's AI and machine learning technologies handle the core tasks of image search model training. To complement Sensei, Adobe has developed several dedicated Elasticsearch plugins:
- Similarity Plugin
- Ranking Plugin
These plugins enhance Adobe's capabilities in image recognition and similar image retrieval, providing the following real-time search functions:
- Face Detection
- Object Detection
- Face Clustering
- Auto Tagging
- Named Entity Recognition
Adobe Stock Machine Learning Image Search Example
Adobe Stock is the flagship platform for Adobe's image search capabilities, with a content library containing over 130 million image assets. Adobe Stock demonstrates the following characteristics:
- Powerful image retrieval capabilities: Precise retrieval based on keywords, attributes, and other information
- Convenient image filtering and sorting functions: Similar image discovery through content, color, composition, and other attributes
- Advanced filtering dimensions: Support for indexing and filtering by price, tags, type, and other fields
Deep Learning Models Working in Synergy with Elasticsearch
The image search technology behind Adobe Stock operates through the synergy of deeply trained machine learning models and the Elasticsearch data storage system. These models employ Convolutional Neural Networks (CNN) to learn deep feature representations (embeddings) of images, mapping these features into similarity metrics in Euclidean distance space to establish clustering relationships among images and their attributes.
Building on this foundation, Adobe has also incorporated a wealth of additional machine learning technologies, making the entire system highly intelligent and delivering users a visually appealing and efficient image search experience.