• Font Size:
  • A
  • A
  • A

Tech Papers

Advantages of Implementing Computer Vision on the Edge 

alwaysAI, Inc.

Introduction

This article explores the strategic and economic advantages of implementing computer vision applications on the edge versus the cloud. It defines the components of edge computing and cloud computing, including the differences between the two computing frameworks and their respective processing power. It then specifies three key advantages of using edge computing versus cloud computing: response times, privacy and security, and cost. 

In the cost-evaluation portion, the article compares the economics of using the cloud vs the edge for two common edge/IoT use cases for computer vision. First, security cameras that use deep neural networks to inference visual data 24 hours a day for a corporate campus, hospital, park, or city. Second, IoT-connected devices that implement vision algorithms and inference data intermittently such as consumer healthcare products and manufacturing processing plants. Finally, the article identifies a real-world case study and calculates the cost of implementing computer vision for both cloud and edge configurations. 

What is Edge Computing?

Edge computing is a form of distributed computing where all computations occur outside of the cloud - i.e., at the “edge” of a centralized server. This function of edge computing is supported by devices that can capture visual data and perform computations locally on the device, closer to the source of the data. Devices that are configured to perform these computations locally don’t need to send data back to the cloud for processing, making that data actionable immediately. This process can be supported by edge devices such as cameras, drones, robots, sensors or edge development environments like the Raspberry Pi or NVIDIA Jetson.

What is Cloud Computing?

Cloud computing, on the other hand, requires devices to send the visual data stream into the cloud for analysis, which then sends back appropriate responses to the users, or the device for further actions. This process relies on data to be transferred via the internet into a central data center. The cloud architecture varies depending on the public cloud provider a developer wishes to employ for her computing needs. In the case of public cloud offerings such as Microsoft Azure, AWS, and Google Cloud Platform (GCP), data enters through a backbone located closest to the source of the data, then this data travels to a central data center for computations. In some cases, the network bandwidth that transfers the immense volume of data traffic bidirectionally to and from the cloud is limited and can not always support the mass volume of data generated by the growing number of data-gathering devices that transfer visual data 24/7. 

Differences in Processing Power

One of the areas where cloud has an advantage over edge is in its flexibility in processing power. The processing power of systems that perform computations on a data-set using machine learning algorithms or deep neural networks (DNN) is measured in TeraFlops. Flops measure a computer’s performance in floating operations per second at a power of 1012. Cloud computing enables systems’ architects and developers to build as many virtual machines as needed to support the processing power of inferencing required by a given visual data stream. In this sense, cloud offers users flexibility in computing power with a pay-per-use model, allowing organizations who use the cloud intermittently to start off small and scale as needed. For such use cases, machine learning algorithms and deep neural networks are hosted in the cloud. The data streams into the cloud, is then processed by the appropriate DNNs, and the results are sent back to the user or the device for further actions. 

The compute power of a DNN on an edge device, however, is limited to TeraFlops available to the edge device. In the case of edge, the DNN’s are delivered directly to the device, and the device performs inferencing and decision making on the edge device in real-time. Though not necessary, these devices can be tethered to the cloud to periodically transfer small amounts of relevant data, eliminating the burden on the network bandwidth.  These devices can perform computations independent of each other and a centralized cloud server, or can be configured to communicate with each other. Today there are over 4 billion edge devices that, when equipped with the appropriate computer vision DNNs, are capable of performing real-time inferencing away from the cloud. This is an enormously untapped source of computational capacity and we’ll see below, has additional advantages.  

Advantages of Edge Computing Vs Cloud Computing

In the explanations above, it is evident that cloud computing is optimal for organizations that require flexibility in processing power. Edge computing, however, is different from cloud and has three major advantages: (1) faster response times with zero to minimal latency; (2) lower security and privacy risks and (3) significantly lower costs associated with setup, inference, and deployment.

The Edge is Faster

The first advantage is that edge devices, in principle, can perform computations closer to the source of the data. This feature enables edge devices to process data quickly and reliably for use cases that require real-time decision making, use cases designed for locations that lack the necessary network infrastructure, and for consumer applications where delays in response can lead to a poor user experience. 

For applications in the cloud, however, latency is determined by the following factors: the Internet service provider (ISP) linked with the data-capturing device, the configuration of the cloud infrastructure, and the hosting location of the centralized data center. A study conducted by ThousandEyes revealed that on average, the further the hosting region from the source of the data, the larger the delay in storing, processing, and execution based on data source. For instance, if the cloud hosting region is Virginia, US, in locations such as North America and Europe, the bi-directional network latency can be between 40ms to 100ms irrespective of the cloud provider. In Asia and Africa, the bi-directional latency is increased up to 200ms! Additionally, the latency in response times can vary dramatically depending on the end-user’s ISP, a feature that can have unfavorable consequences for systems that require reliable response times. 

Unlike cloud computing, the edge has no dependency on a central data center and network bandwidth and eliminates latencies in response times. This procedure is optimal for use cases that require immediate action based on presented data. Imagine an IoT connected college campus that identifies gun threats on campus. How long are you willing to wait for the system to determine that an individual is armed, before reporting this to the proper authorities? For this case, sending data from hundreds of cameras on a college campus into the cloud may be prohibitive with respect to the network and lead to delayed response times. Even if the network on campus were temporarily disabled, an edge device can still immediately identify the threat and report the threat to appropriate campus authorities. Similarly, in the case of autonomous vehicles or robotic surgeons, a network disruption or milliseconds of latency can drastically affect the outcome of the use case. 

Edge computing in general, can also be network independent. This feature enables computations and inferencing to occur reliably in various remote locations that lack the necessary network infrastructure. This enables the development of new IoT technologies in industries such as agriculture, waste management, and human search and rescue technologies. When the dependency on a network is removed and instead DNNs are deployed to the edge, one can build IoT devices that streamline agriculture processes in remote farms, or robots that go far out in the ocean to identify and remove waste, and search and rescue drones that can be deployed to locations hit with a natural disaster. All of these edge use cases require little to no access to servers in order to perform their key functions and can operate in locations underserved by traditional internet service providers’ wireline infrastructure. 

Finally, the speed and efficiency in edge computing can also be a benefit to consumer products such as smart assistants and home-robots that leverage AI inferencing as a means of communicating with the users. Consumer demand dictates that information processing and response be available immediately when a request is made. The dependency on cloud and network latency can have significantly negative consequences for user experience. Edge computing empowers systems designed for such use cases to reliably process mass streams of data, and make real time decisions without the latency that comes with cloud computing. 

Privacy & Security

The second advantage of edge computing is privacy; security factors are also considered. Certainly, if a centralized cloud server is breached, then all the data is potentially exposed. Edge computing, however, is a form of distributed computing, where if one edge device is compromised, only the data associated with that specific edge device is exposed. Even when conjoined, the edge configuration distributes data collection and processing across multiple devices, and consequently distributes the risk of exposure. 

In the 21st century, we’ve seen cyber attacks against well known cloud companies such as DropBox where 7 million user passwords were compromised. Similarly, there have been other cyberattacks against well-known organizations such as Adobe, Target, eBay and Linkedin where the common theme of such cybercrimes is theft of user data. See chart below for the number of users’ data that has been compromised as a result of such crimes. 

Cyber Attacks in 21st CenturyThe top threats that cloud companies face are “Insecure Interfaces and API's, Data Loss & Leakage, and Hardware Failure—which accounted for 29%, 25% and 10% of all cloud security outages respectively” according to The Cloud Security Alliance. It must also be noted that cloud providers host data for hundreds and thousands of companies, where even a single breach to the cloud can compromise massive volumes of data, a phenomena characterized as “hyperjacking”. To prevent this, public cloud computing services typically have strong security systems in place. However, in order to be stored and processed in the cloud, this data needs to travel the public internet. This can pose the risk of sensitive data being intercepted and breached in its route to the cloud system. The risk in using cloud services for computer vision inferencing is not just the risk of exposure in the event the centralized server is breached, but also in the process of sending the data to the cloud. It's also important to highlight that an edge device is also prone to security attacks. Common security threats against edge include authentication, insecure communications, and the device’s maintenance mode access. These risks are mitigated by the limited footprint of an edge device as compared to a cloud computing platform. 

An important consideration in the design and execution of consumer products, particularly ones that involve collecting visual data is, privacy. Consider systems’ use cases that interact and respond to a user in real time such as a fall-detection device enabled with pose estimation that identifies if an elderly person has fallen. Using edge computing, these systems can be installed inside of a person’s home, process real-time data and inform the proper authorities without streaming the user’s video data into the cloud. An edge device can perform all of its processing closer to the users, entirely disconnected from a centralized server. When data (real-time video feed coming from inside of a person’ home) is processed on the edge, that data does not leave the data source (a person’s home) even when an event occurs (such as someone falling). The edge device can make appropriate decisions without ever transferring the video data out of a person’s home. 

A similar case can be made for health, financial, and retail markets where sensitive or private information must be shared between users and the system privately for optimal user experience. Using cloud computing for such industries is risky, but it must be noted that an edge device is also susceptible to hacking. The security factor with edge is in the configuration of these devices. Although still risky, the edge framework enables users of these devices to distribute the data across multiple devices, consequently distributing the risk of exposure. Where critical processes and data need to be protected, using edge computing is a far more optimal solution with respect to ensuring privacy and security than cloud.

Economics of Edge vs Cloud

Finally, there is the economic value of using edge as opposed to cloud for deep learning computer vision based applications - and this is where you see the real payoff. Cloud companies offer flexible pricing and charge for inferencing per endpoint, per minute. This is beneficial for organizations that would prefer to pay for their data storage and processing based on need. However, for organizations that demand large amounts of data processing, real-time and secure inferencing such as an IoT connected Smart City, or a hospital that processes data 24 hours a day every day, it is simply economically unscalable to perform all computations on the cloud server.  

The following section calculates the cost differences of deploying and inferencing computer vision algorithms on the edge versus the cloud. The costs of implementing DNNs and machine learning models can be broken down into the following categories: network costs, electricity, cost of hardware, cost of Inferencing, labor, and additional costs associated with setting up a local server to extract sensitive or private data before sending it to the cloud. For this exercise, we will primarily focus on the cost of inferencing in the cloud vs inferencing using an edge device. This will be evaluated based on two categories of IoT use cases. First, we’ll analyze the cost of inferencing data from surveillance cameras on a corporate campus, a hospital, a park or a city which requires 24 hours a day of inferencing and data processing. Second, we’ll evaluate the cost of inferencing IoT devices that intermittently connect to the cloud such as in the case of manufacturing plants that inference visual data only during business hours (10 hours a day, 20 days a month), retail stores that also operate within a shorter time window, and devices such as rescue drones that are designed for short-term operations.  

Inferencing Costs in the Cloud (24 hour Surveillance)

Google cloud platform hosts machine learning models that can be used for image and video analysis. For this platform, there are additional costs associated with Compute Engine instances, and Cloud Storage etc. In the Google Cloud platform (GCP), the hourly cost of inferencing for Object Tracking on a real time video feed is $0.42. For a single camera, such as the ones for a connected hospital where real time video data is sent to the cloud for inferencing 24/7, the monthly cost for predictions on one camera feed could be $302.40. In the case of a hospital which will likely have at the minimum 20 cameras, the monthly cost of processing this on the cloud using GCP could be $6,048. A larger hospital, or an IoT connected park with 65 cameras performing real-time inferencing 24/7 could reach upto $19,656 a month.  

A lower cost option for computer vision inferencing, which does not include cloud storage, would be Amazon Elastic Inference service which cost $0.29 an hour for real time video inferencing. Elastic Inference attaches a low cost GPU-powered accelerator to enable inferencing on deep learning ML algorithms. We are only evaluating the cost of inferencing using these services. In the case of a single camera inferring data for 24/7, a monthly cost for this service can be up to $208.80, $4,176 for 20, and up to $13,572 for 65 devices.  See chart below for a detailed breakdown of costs using GCP, Amazon Elastic Inference, and alwaysAI, a computer vision software that enables the deployment of computer vision algorithms on edge devices. 

CLOUD COST ANALYSIS: 24 HOURS/ 7 DAYS A WEEK REAL TIME OBJECT DETECTOR?

Average Cost per Month?

GCP AutoML Pricing?

$0.42/ Hour/ Endpoint?

Amazon Elastic Inference

$0.29/ Hour/ Endpoint?

alwaysAI?

Flat Rate?

1 Endpoints?

$302.40/month

$208.80/month

$0?

20 Endpoints?

$6,048/month

$4,176/month

$99/month?

65 Endpoints?

$19,656/month

$13,572/month

$299/month?

 

Inferencing Costs in the Cloud (10 hours of Inferencing)

At $0.29 an hour, one might assume that if real-time inferencing occurs for only 10 hours a day, 20 days a month then cloud computing may provide a scalable solution for manufacturing plants, or IoT devices that connect to the cloud intermittently for inferencing. The costs of using cloud for such a configuration could be economically acceptable in the short-term, however, the costs in the cloud are recurring and can become very expensive after the course of 1, 2, 3, or even 4 years (see case study below). 

CLOUD COST ANALYSIS: 10 HOURS/DAY, 5 DAYS/WEEK REAL TIME OBJECT DETECTOR?

Average Cost per Month?

GCP AutoML Pricing?

$0.42/ Hour/ Endpoint?

Amazon Elastic Inference

$0.29/ Hour/ Endpoint?

alwaysAI?

Flat Rate?

1 Endpoints?

$84/month

$58/month

$0?

20 Endpoints?

$1,680/month

$1,160/month

$99/month?

65 Endpoints?

$5,460/month

$3,770/month

$299/month?

 


At this rate, the cost for using Amazon Elastic Inference can be upto $45,240.00 for a year and $180,960.00 for 4 years. These costs may seem considerably low compared with the cost of inferencing data for 24 hours a day, however, edge computing is still significantly lower. 

alwaysAI’s edge software, has a compelling freemium offer that enables computer vision algorithms on up to 3 devices. For organizations that wish to deploy to more than 3 devices, the starting cost of a subscription will be in the range of $99 to $299 per month depending on the number of licensed developers, deployed edge devices and additional services like model-retraining.

Cost by Processing Power

Another way of comparing costs associated with cloud inferencing vs edge is by considering the costs of processing power. The cost of GPUs is inherent in the inferencing costs above. However, the pricing below reveals that even when the processing power on the edge is equivalent to the processing power in a cloud GPU, the edge costs are still much lower. In order to understand this, it's important to look at costs associated with computational power per GPU, and the GPU’s cost by TeraFlops (TFLOPS) in the cloud. Amazon Sagemaker offers 32 TFLOPS for $.48 an hour. At this rate, the cost of this plan can scale up to $342.72 monthly and $4,112.64 yearly.  For 16 TFLOPS, Amazon Sagemaker’s costs are $0.34 an hour, which can scale to $242 monthly, and  $2,903 yearly. 

Comparably, an edge device such as the Qualcomm RB3 is a one-time investment of $499 and provides 15 TFLOPS of computational power. Purchasing two or even three RB3’s would provide the same level of computational power as the cloud, yet at a fraction of the computing costs. 

CLOUD COST ANALYSIS: PROCESSING POWER

System?

TFLOPS?

Cost ?

Cost per Month?

Cost per Year/GPU?

Hardware

+

alwaysAI ?

Sagemaker (32)?

32 TFLOPS?

~$0.48?

$342.72?

$4,112.64?

-?

Sagemaker (16)?

16 TFLOPS?

~$0.34?

$242?

$2,903?

-?

Qualcomm RB3?

15 TFLOPS?

N/A

N/A?

N/A

$449 ?+

$99/month

NCS 2 + Raspberry Pi?

4 TFLOPS?

N/A

N/A

N/A

$178+

$99/month

NVIDIA Jetson?

0.5 TFLOPS?

N/A

N/A

N/A

$399+

$99/month

 


A Hybrid Pricing Structure: Edge and Cloud 

In evaluating the pricing structures and various pricing tiers of these cloud services, an organization may conclude that although it may be cost effective to perform much of the real-time inferencing on the edge, a use case may demand the data be transferred to the cloud for additional inferencing even after the edge has performed much of its computations. This leads us to a ‘hybrid’ pricing structure where much of the processing and computation takes place on the edge, but the processed data is intermittently streamed into the cloud for further analysis. In this case, we can assume that all edge devices upload data into the cloud every 24 hours. The monthly cloud costs for 20 device streaming data to the cloud once a day for 1 hour  can be $174 with Elastic Inference. Note, the costs in the table below are still accounting for inferencing in the cloud, the cost of storing data only, are even lower (see tables in the case study below). 

Hybrid: Edge + Cloud Cost Analysis for 20 Devices

Hardware

Cost of Hardware (x20)

alwaysAI Monthly Subscription

Elastic Inference

 (1 hour)

Total Monthly Cost

Qualcomm RB3 $449

$8,980

$99

$174

$9,253

NCS 2 $99 + Raspberry Pi $79

$3,560

$99

$174

$3,833

NVIDIA Jetson

$399

$7,980

$99

$174

$8,253

 


This reveals a more realistic pricing structure of using cloud intermittently, while using the power of edge computing for much of the real-time analysis. Such a structure empowers users to harness the instant compute power of the edge while still employing the flexibility and storage power of the cloud for a hybrid model of cloud and edge computing.

Case Study

Lets now calculate the cost of setting up and running a computer vision application for a real world use case, with and without alwaysAI. We will build and price a computer vision project for a local county train station. In an effort to ensure greater security for travelers, the local train station has authorized the implementation of computer vision to identify prohibited objects before they are carried inside of a train. The objective of the computer vision application of this train station is to 1) identify forbidden items being carried into trains such as a knife, scissors etc. 2) immediately report to the station authorities that such an object has been identified. For this section, we’ll cover the set-up costs in the cloud (without alwaysAI) and on the edge (with alwaysAI). Next, we’ll calculate the recurring and accumulative cost for the span of 1, 2, and 3 years for both configurations. 

General project assumptions are as follows:

  1. The train station operates 24 hours a day, everyday. Therefore, the computer vision model will process data from 20 camera feeds for 24 hours every day.

  2. The Object Detection Model to be used is: “ssd_mobilenet_v1_coco_2018_01_28”

  3. There are no additional costs associated with training a model. 

  4. All Labor costs are based on in-house labor costs of $60/hour. The engineer’s average working day is 8 hours a day. The labor costs are calculated for a single engineer working full time on the project. 

  5. Computer Vision configuration shall be delivered in 90 business days

  6. All major labor costs will be associated with set up only. The maintenance costs associated with the configuration are classified as operational and unaccounted for in the yearly cost comparisons. 

  7. The yearly cost analysis will assume that the number of devices has not changed

  8. The cost analysis does not account for cost of ISP, or electricity as these vary based on region 

  9. HD Video recording speed shall be 6mbps

Cloud Inferencing Assumptions:

  1. Cloud Cost analysis will be computed using Amazon Elastic Inference ($.29/hour). 

  2. A local server will be needed in order to extract sensitive data prior to sending data back to the cloud. 

  3. Throughput and data size of each record are stable and constant throughout the day.

  4. Amazon S3 will be used for Data Hosting services  (Note- that Amazon Kinesis for real time video stream also uses S3 for storage)

Edge Inferencing Assumptions:

  1. Edge IoT Device: NVIDIA Jetson Nano

  2. Object detection model will be deployed directly to the edge device

  3. Edge devices are network independent and only report back to Train Station authorities once a forbidden object is identified.

  4. Some data will be transferred and stored in the cloud (Amazon S3) for further analysis.

Based on the aforementioned assumptions, see below for the costs of configuring computer vision for a train station with 20 sources of visual data without using alwaysAI. For the cloud set-up, we’ll allow for 45 days for setting up the device, the cloud servers, as well as the configuration of a local server to extract sensitive data, we will assume that QA and inferencing from all 20 camera feeds will begin after 45 days. Note that with this configuration, additional time may be needed to support the configuration of a local server.  

CLOUD COST ANALYSIS: WITHOUT ALWAYSAI

Expenses

Cost/Unit

Total Cost 

S3 Storage

Set up costs

(First 45 days)

Cameras: Megapixall Wi-Fi Smart Home Bullet Camera (20 ct)

$99

$1,980

128 Gb/ 6 mbps

Local Server to extract Sensitive video data: Jackal Pro 1U

$2,999

$2,999

-

Total Hardware Cost

-

$4,979

-

Labor x 45 days

Labor Costs (1)

$21,600.00

$21,600.00

-

Total Set Up Cost

$26,579

-

Cloud Inferencing

Amazon Elastic Inference x 20 endpoints x 45 days

$0.29/ hour

$6,264.00

$5,355.11/45 days

Labor x 45 days

Labor Costs (1)

$21,600.00

$21,600.00

 

Total Set Up Costs + Inferencing + Cloud Storage

$51,444.00

-

 


When using an edge device and an edge based software such as alwaysAI, the costs of setting up are slightly higher, but the payoff of using edge computing can be seen when the inferencing occurs over the course of a year. Additionally, because this configuration permits all computations to occur locally at the edge, a local server is not required, hence, realistically, the set up time could be much shorter than 90 days, potentially reducing the labor costs associated with the set up. Modifying the models is also very simple using the alwaysAI CLI. Note, based on the assumptions and requirements above, some data will need to be transferred to the cloud for further analysis. For this, we’ll assume that 2,560 minutes of video data will be sent to the cloud every month for further analysis. 

EDGE COST ANALYSIS: WITH ALWAYSAI

Expenses

Cost/Unit

Total Cost ?

Cloud Storage Costs

Device: NVIDIA Jetson (20 ct)

$399

$7,980

$11.43/month

Cameras: Megapixall Wi-Fi Smart Home Bullet Camera (20 ct)

$99

$1,980

128 Gb/ 6 mbps

Total Hardware Cost

-

$9,960

 

Labor Costs (1)

$43,200.00

$43,200.00

 

Software Cost (alwaysAI) $99/month

$297

$297

 

Total Set Up Cost

$63,428.43

 
 


After the setup is complete, the yearly costs associated with cloud is as follows:

CLOUD COST ANALYSIS: YEARLY COSTS

Cost of Inferencing by Year

Accumulative Cost for Cloud Services

Cameras ?

Local GPU

Hosting Data ($3,776.57/Month)

Total Cost

1 Year

$50,112

$1,980

$2,999

$45,318.84

$100,409.84

2 Year

$100,224

$1,980

$2,999

$90,637.68

$195,840.68

3 Year

$150,336

$1,980

$2,999

$135,956.52

$291,271.52

 


After the setup is complete, the yearly costs associated with edge computing using alwaysAI is as follows. Note- that the cost of hosting data is subject to change based on the size of the data transferred to the cloud. To calculate the cost based on your record size, input the GB of data in the Amazon S3 cost calculator and update the hosting cost column for an updated total cost. 

EDGE COST ANALYSIS: YEARLY COSTS

Cost of Inferencing by Year

Accumulative Edge Software Costs

Cameras ?

Edge Device Cost

Hosting costs $11.43/month

Total Cost

1 Year

$1,188.00

$1,980

$7,980

$137.16

$11,285.16

2 Year

$2,376.00

$1,980

$7,980

$274.32

$12,610.32

3 Year

$3,564.00

$1,980

$7,980

$411.48

$13,935.48

 


Which configuration would you choose if you were representing the train station above? The calculations above reveal the cost of setting up computer vision and the cost of inferencing for an ongoing period of time. The analysis reveals that the cost of performing computer vision on the edge is fractional when evaluated over the course of a few years even when some of the analyzed data is stored in the cloud. With a software that implements deep learning vision algorithms to the edge being a fraction of the cost of cloud processing, the initial hardware costs simply become an inexpensive fixed asset. 

Hardware industry pioneers such as NVIDIA, Intel, and Qualcomm have made considerable innovations in AI on the edge space. If every single one of these devices is enabled with deep learning AI algorithms, then the world as we know it, would be remarkably transformed. The aggregate data processing power of these devices will revolutionize the world of technology, agriculture, finance, smart cities, and most importantly human-wellness. When we tap into the potential of these intelligent edge devices, we’ll have designed tools and systems to help us understand our world and reconstruct it for the better. With alwaysAI, users can cost-effectively enable these smart IoT devices with the power of computer vision on the edge. alwaysAI is proud to be a part of the phenomena and we are excited to unleash the value of the edge to individuals and companies across the world.

 

Search AIA:


Browse by Products:


Browse by Company Type: