👋Welcome to Lens AI
Lens AI is a Model and Data observability platform build focused on Edge devices.
Last updated
Lens AI is a Model and Data observability platform build focused on Edge devices.
Last updated
Artificial intelligence (AI) is increasingly being deployed on edge devices, such as smartphones, cameras, and sensors. This is because edge devices can collect and process data more quickly and efficiently than cloud-based systems. However, AI models on edge devices are also more vulnerable to cyberattacks and data corruption.
Data and model observability are crucial components of the MLOps cycle. While observability practices are well-established in the cloud environment, they present unique challenges in edge model deployment. Ensuring effective model and data observability at the edge is more complex compared to the cloud, necessitating innovative solutions to address these challenges.
While data and model monitoring has reached maturity within the realm of cloud AI, its adoption in Edge AI remains relatively limited.
The following are the key differences between Cloud AI and Edge AI.
1. Location: Cloud AI processes data in remote data centers, while Edge AI operates locally on devices.
2. Latency: Cloud AI can have higher latency due to data transfer, while Edge AI offers real-time processing.
3. Privacy: Edge AI preserves data privacy since it doesn't rely on external servers.
4. Bandwidth: Edge AI reduces the need for constant high-bandwidth connections.
5. Scalability: Cloud AI can scale more easily for large-scale tasks, while Edge AI is limited by device capabilities.
6. Use Cases: Cloud AI suits data-intensive, centralized tasks, while Edge AI is for distributed, on-device applications.The AI-equipped Edge device has been deployed into the real world, one of the primary responsibilities of AI/ML Engineers is as follows:
Ensure the model operates as intended.
Identify and mitigate potentially harmful, uncontrolled scenarios and bugs.
Enhance the device's intelligence and tackle edge cases effectively.
Within the domain of Model and Data Monitoring, AI/ML Engineers grapple with concepts such as Data Drift and Model Drift or Concept Drift. In addition to these, it is imperative to monitor system-level and application-level metrics, as they play a pivotal role in ensuring the desired AI functionality on the device.
Models are analogous to cars in the sense that their accuracy tends to decrease once they are out in the real world, just like cars depreciate in value once they leave the showroom floor.
Monitoring serves as a valuable tool for addressing various critical questions:
1. What has caused a decline in prediction accuracy?
2. Which variations in the data should be integrated into the training process to enhance the model's robustness?
3. Do the model's predictions align with the overarching business goals and objectives?
4. How effectively does the model handle outliers in a production environment?
5. To what extent does the training data resemble the data observed in production?
6. Is my model robust against model poisoning and adversarial attacks?
Detecting and resolving model-related issues becomes notably challenging without diligent monitoring, hindering model observability. Effective monitoring enables us to conduct root cause analyses and swiftly identify and rectify bugs, significantly reducing the time taken for troubleshooting.
Monitoring can be broadly classified as Functional Monitoring and Operational Monitoring.
Functional monitoring primarily focuses on assessing input data quality, monitoring model performance, and evaluating the model's output.
Operational monitoring involves tracking system and resource-level metrics related to model inference, and watchdog timers ideally handle the majority of these operational monitoring functions.
The key tasks for
Analyze the input data
Analyze the model behavior for the incoming data
Conducting the aforementioned analysis at the edge presents several challenges, whereas performing analysis for models deployed in the cloud is comparatively straightforward.
The key challenges are as follows.
Limited data storage capacity: Edge devices operate with constrained and costly data storage solutions, making it challenging to match the flexibility and cost efficiency of cloud-based data storage. So at the production scale, it is practically not possible to build storage for the incoming data on the edge.
Massive volumes of sensor data: The data generated by sensors may exhibit variability depending on sensor types. When dealing with multimedia data, such as images, videos, and audio, it has the potential to rapidly escalate into a substantial volume. Example: A single image sensor operating at 30 FPS with 1.2 MP resolution and 24-bit color depth without compression can generate around 8.6 TB of data in a day.
Limited and Intermittent connectivity: Edge devices face diverse connectivity challenges depending on their field deployment and the communication technologies employed. For instance, LoRa-based sensors deployed in the field exhibit distinct bandwidth constraints compared to their 4G counterparts. This makes it impractical to transfer the data to the cloud.
Expensive data transfer: Transferring data at the edge involves costs and sometimes, it may not be economically feasible to transmit all the data from sensors to the data warehouse. This cost will explode quickly with the scale of sensors Therefore, it's crucial to determine how data should be selected for monitoring purposes.
Choosing the Right Data: It is important to select the right data points to monitor at Sensor Level, Model Level, and System Level. This helps in Identifying the root cause of many model-related problems and improves the time to resolution from months to weeks.
Computing Constraints: Edge devices are constrained in computing power, which makes it difficult to employ complex algorithms to deal with active learning on the device to avoid model and data drift.
By monitoring AI models and data on edge devices, organizations can help to ensure that their AI systems are operating as intended and that they are not vulnerable to cyberattacks.
Some of the benefits of monitoring AI models and data on edge devices:
Improved performance: Monitoring can help to identify and address problems that are affecting the performance of AI models, such as model drift and data corruption. This can lead to improved accuracy and reliability of AI-powered applications.
Increased security: Monitoring can help to identify and address security threats, such as model poisoning and adversarial attacks. This can help to protect AI systems from being compromised.
Reduced costs: Monitoring can help to identify and address problems early on before they cause major disruptions or outages. This can help to reduce the costs of troubleshooting and repairs.
An AI-equipped device without monitoring the model and data is similar to a car without brakes – Just as brakes are essential for controlling and ensuring the safety of a car's speed and direction, monitoring is crucial for ensuring the desired functionality, accuracy, and security of AI systems.
