📝Step 2: Integrating Lens AI Cpp Profiler on Edge

The current version of the Lens AI supports vision data in the next releases supports audio , timeseries and text data.

Lens AI Cpp profiler should be integrated on the edge device, where the inference happens.

Lens AI Cpp Profiler

Build & Install the Cpp library on the device there are also prebuilt packages for Ubuntu x64-86 and Arm architectures.

https://github.com/lens-ai/lensai_profiler_cpp.git 
cd lensai_profiler_cpp 
mkdir build
cmake .. -D CMAKE_BUILD_TYPE=RELEASE
make install
ldconfig

There are also pre-built ubuntu docker images available on docker hub.

• docker pull vsnm/lensai_profiler_cpp:latest

• docker run -it --rm vsnm/lensai_profiler_cpp

Once the library is built and successfully installed then integrate the profiling code in your inference script

Define the configuration file based on the output from the previous step of computing the thresholds.

Metric		Description
sampling		Sampling data where model is most uncertain.
	MARGINCONFIDENCE	Margin Confidence of the classes are above the threshold then sample those data points
	LEASTCONFIDENCE	Least Confidence of the classes are above the threshold then sam
	RATIOCONFIDENCE	Ratio Confidence of the classes are above the threshold then sam
	FILEPATH	Path in which to save the metrics and samples
Image		Image related metrics.
	NOISE
	BRIGHTNESS
	SHARPNESS
	CHANNELS
	HISTOGRAM
	FILEPATH	Path to image related metrics and samples
model		Model Metrics.
	FILEPATH	Path to the model related metrics and samples

Please keep all the metrics that need to be computed, 
if deleted the corresponding metrics are not computed, 
use NaN to disable sampling and just compute the metrics. 

[sampling] 
MARGINCONFIDENCE = 0.01, 0.9 
LEASTCONFIDENCE = 0.01, 0.9
RATIOCONFIDENCE = 0.01, 0.9 
ENTROPYCONFIDENCE = 0.01, 0.9 
filepath = /tmp/samples/ 
[image] 
CHANNELS = 3 
NOISE = 3, 14 
BRIGHTNESS = 23, 255 
SHARPNESS = 30, 255 
MEAN = NaN 
HISTOGRAM = NaN 
filepath = /tmp/imgstats/ 
[model] 
filepath = /tmp/modelstats/  

The metrics always a lower and upper threshold is defined to capture the samples at the tails of the distribution. default values for the confidence are 0.01 and 0.9 meaning it samples the images that model is 99.9 % confident or above or below 10 % confident.

define the frequency at which you want to save the metrics

int saveIntervalSec = 1; 
int img_channels = 3; // Number of Channels

  std::string modelName = "Cat_dog_classfier_0.1";

  ImageProfile image_profile(configFile, saveIntervalSec, channels);
  ModelProfile model_profile(modelName, configFile, saveIntervalSec, channels);
  ImageSampler image_sampler(configFile, saveIntervalSec);

In the inference part after the inference, you can ass the following line to start profiling

std::cout << "profiling image profile" <<std::endl;
image_profile.profile(frame, true);
 
std::cout << "profiling model profile" << std::endl;
model_profile.log_classification_model_stats(10.0, top_results);
    
std::cout << "profiling samper" << std::endl;
 image_sampler.sample(top_results, image, true);