Tag Archives: Alibaba Cloud

The Alibaba Hanguang 800 (含光 800) AI NPU Explained!

The Alibaba Hanguang 800 (含光 800) AI NPU Explained!

At the Apsara Computing Conference 2019, Alibaba Group unveiled details of their first AI inference NPU – the Hanguang 800 (含光 800).

Here is EVERYTHING you need to know about the Alibaba Hanguang 800 AI inference NPU!

Updated @ 2019-09-27 : Added more details, including a performance comparison against its main competitors.

Originally posted @ 2019-09-25

 

What Is The Alibaba Hanguang 800?

The Alibaba Hanguang 800 is a neural processing unit (NPU) for AI inference applications. It was specifically designed to accelerate machine learning and AI inference tasks.

 

What Does Hanguang Mean?

The name 含光 (Hanguang) literally means “contains light“.

While the name may suggest that it uses photonics, that light-based technology is still at least a decade from commercialisation.

 

What Are The Hanguang 800 Specifications?

Not much is known about the Hanguang 800, other than that it has 17 billion transistors, and is fabricated on the 12 nm process technology.

Also, it is designed for inferencing only, unlike the HUAWEI Ascend 910 AI chip which can handle both training and inference.

Recommended : 3rd Gen X-Dragon Architecture by Alibaba Cloud Explained!

 

Who Designed The Hanguang 800?

The Hanguang 800 was developed over a period of 7 months, by Alibaba’s research unit, T-Head, followed by a 3-month tape-out.

T-Head, whose Chinese name is Pintougehoney badger in English, is responsible for designing chips for cloud and edge computing under Alibaba Cloud / Aliyun.

Earlier this year, T-Head revealed a high-performance IoT processor called XuanTie 910.

Based on the RISC-V open-source instruction set, 16-core XuanTie 910 is targeted at heavy-duty IoT applications like edge servers, networking gateways, and self-driving automobiles.

 

How Fast Is Hanguang 800?

Alibaba claims that the Hanguang 800 “largely” outpaces the industry average performance, with image processing efficiency about 12X better than GPUs :

  • Single chip performance : 78,563 images per second (IPS)
  • Computational efficiency : 500 IPS per watt (Resnet-50 Inference Test)
Hanguang 800 Habana Goya Cambricon MLU270 NVIDIA T4 NVIDIA P4
Fab Process 12 nm 16 nm 16 nm 12 nm 16 nm
Transistors 17 billion NA NA 13.6 billion 7.2 billion
Performance
(ResNet-50)
78,563 IPS 15,433 IPS 10,000 IPS 5,402 IPS 1,721 IPS
Peak Efficiency
(ResNet-50)
500 IPS/W 150 IPS/W 143 IPS/W 78 IPS/W 52 IPS/W

Recommended : 2nd Gen EPYC – Everything You Need To Know Summarised!

 

Where Will Hanguang 800 Be Used?

The Hanguang 800 chip will be used exclusively by Alibaba to power their own business operations, especially in product search and automatic translation, personalised recommendations and advertising.

According to Alibaba, merchants upload a billion product images to Taobao every day. It used to take their previous platform an hour to categorise those pictures, and then tailor search and personalise recommendations for millions of Taobao customers.

With Hanguang 800, they claim that the Taboo platform now takes just 5 minutes to complete the task – a 12X reduction in time!

Alibaba Cloud will also be using it in their smart city projects. They are already using it in Hangzhou, where they previously used 40 GPUs to process video feeds with a latency of 300 ms.

After migrating to four Hanguang 800 NPUs, they were able to process the same video feeds with half the latency – just 150 ms.

 

Can We Buy Or Rent The Hanguang 800?

No, Alibaba will not be selling the Hanguang 800 NPU. Instead, they are offering it as a new AI cloud computing service.

Developers can now make a request for a Hanguang 800 cloud compute quota, which Alibaba Cloud claims is 100% more cost-effective than traditional GPUs.

 

Are There No Other Alternatives For Alibaba?

In our opinion, this is Alibaba’s way of preparing for an escalation of the US-Chinese trade war that has already savaged HUAWEI.

While Alibaba certainly have a few AI inference accelerator alternatives, from AMD and NVIDIA for example, it makes sense for them to spend money and time developing their own AI inference chip.

In the long term, the Chinese government wants to build a domestic capability to design and fabricate their own computer chips for national security reasons.

Recommended : The HUAWEI Trump Ban – Everything You Need To Know!

 

Recommended Reading

Go Back To > Business + Enterprise | Home

 

Support Tech ARP!

If you like our work, you can help support our work by visiting our sponsors, participating in the Tech ARP Forums, or even donating to our fund. Any help you can render is greatly appreciated!


3rd Gen X-Dragon Architecture by Alibaba Cloud Explained!

At the Apsara Conference 2019, Alibaba Cloud announced that they will be introducing the 3rd Gen X-Dragon Architecture for their cloud servers!

Here is a quick PRIMER on the new 3rd Gen X-Dragon Architecture!

 

What Is X-Dragon?

X-DragonShenlong in Chinese – is a proprietary bare metal server architecture developed by Alibaba Cloud for their cloud computing requirements.

Built around a custom X-Dragon MOC card, it delivers what Alibaba Cloud calls Elastic Compute Service (ECS) capability in a bare metal server.

The ECS bare metal instances it offers combine the benefits of bare metal servers, and virtual machines.

For example, it offers direct access to CPU and RAM resources without virtualisation overheads that bare metal servers offer, with the instant deployment and image migration capabilities of virtual machines.

The downsides? ECS bare metal instances, once deployed, cannot be upgraded or downgraded. In addition, if there is a hardware failure, a failover occurs and the data remains stored in the instance’s storage drives.

 

What’s New In The 3rd Gen X-Dragon Architecture?

Basically – SPEED.

[adrotate group=”2″]

According to Alibaba Cloud, the 3rd Gen X-Dragon architecture is able to increase Queries Per Second (QPS) by 30% and decrease latency by 60% in e-commerce scenarios.

In tandem, they also announced the 6th Gen ECS instance, which delivers a 20% boost in computing power, a 30% reduction in memory latency, and a 70% reduction in storage IO latency.

Not new, but also important is the fact that because it is cloud-native by design, it eliminates power wastage from idle bare metal servers. Alibaba Cloud claims that alone reduces the unit computing cost by 50%.

 

3rd Gen X-Dragon Architecture Availability

Alibaba Cloud will start rolling out the 3rd Gen X-Dragon architecture upgrade to millions of their cloud servers around the world from 2020 onwards.

 

Recommended Reading

Go Back To > Business + EnterpriseComputer | Home

Support Tech ARP!

If you like our work, you can help support our work by visiting our sponsors, participating in the Tech ARP Forums, or even donating to our fund. Any help you can render is greatly appreciated!


Fine For Scratching Nose A Wake-Up Call On AI Surveillance!

The recent case of a Chinese driver getting fine for scratching his face is a funny example of current AI surveillance technology. Yet it is also a wake-up call on the dangers of pervasive AI surveillance by the state.

 

Fined For Scratching Nose By AI Surveillance System!

A Jinan resident, Mr. Liu, was driving his car in the eastern Shandong province, when he raised his hand to touch his face. Most of us unconsciously do that 2 to 5 times per minute!

Unbeknownst to him, one of the many AI surveillance cameras in the city noticed his action, and issued him a fine of 50 yuan* and 2 demerit points for “driving while holding a phone“.

* Approximately $7.25 / £5.70 / €6.50 / RM 30

The Jinan AI surveillance system also sent him this screenshot of his traffic violation, as captured at 7:20 AM on 20 May 2019.

Just like many automated systems (looking at you, Facebook and Google!), there was no way to dispute the charge. Mr. Liu tried to sort out the situation over the phone, but “no one would help him“.

He only got justice by appealing to the court of public opinion on Sina Weibo, where his post went viral. Only then did the Jinan traffic police department take notice and investigate his complaint.

Two days later, they cancelled his ticket after confirming that he was only touching his face, and not actually using a phone while driving.

 

AI Surveillance In Chinese Cities

China has been working hard at developing smart cities, as part of their social engineering efforts to quell political dissent and encourage Chinese citizens to “behave properly”.

There are already over 170 million surveillance cameras across China, with a projected 400 million surveillance cameras installed by next year. And they are all controlled by AI surveillance systems.

Such extensive surveillance coverage has allowed the Chinese government to detect crimes and punish their citizens for them. It also feeds the new Social Credit System – a national reputation system that assess the economic and social reputation of every Chinese citizen and business.

However, such pervasive surveillance has led to serious privacy implications for the Chinese citizenry. Anyone who wants to understand the power, allure and dangers of AI surveillance should watch the TV series, Person of Interest.

 

The Dangers Of AI Surveillance

While AI surveillance technology is now quite incredible, this case has exposed its vulnerabilities and limitations.

  1. Human oversight is still necessary, because AI surveillance is not accurate enough to detect false positives.
  2. It may be tempting to make the AI surveillance system the judge, jury and executioner, but such systems need to implement the principle of “guilty beyond a reasonable doubt“, and that means ignoring anything that is not close to a 99.9% match.
  3. There should be an appeal system in place. It took a viral social media post to alert the Jinan traffic police department to the mistake.
  4. There is also the question of personal data security. Can the government securely store the data, without unsanctioned or illegal access? How long should they store the information before they are deleted?

 

Alibaba Cloud + The Malaysia City Brain

Alibaba Cloud is one of the chief architects of Chinese smart city initiative and AI surveillance capabilities with their ET City Brain that runs on their Tianchi Platform.

Last year, Alibaba Cloud announced their collaboration with the Malaysia Digital Economy Corporation (MDEC) to introduce the Malaysia City Brain.

The first phase of the Malaysia City Brain will kickstart with 382 AI traffic cameras at 281 traffic light junctions in Kuala Lumpur.

Although the Malaysian government is ostensibly implementing the Malaysia City Brain to “optimise the flow of vehicles and timing of traffic signals“, it is really a short step to the Chinese model of population and crime surveillance.

 

Recommended Reading

Go Back To > Cybersecurity| Enterprise | AutomotiveHome

 

Support Tech ARP!

If you like our work, you can help support our work by visiting our sponsors, participating in the Tech ARP Forums, or even donating to our fund. Any help you can render is greatly appreciated!


Dr. Min Wanli : Alibaba Cloud’s ET City Brain & Tianchi Platform

At the MDEC-Alibaba Cloud announcement of their collaboration to implement the Malaysia City Brain, Dr. Min Wanli gave a quick overview of the ET City Brain and the Tianchi platform for crowd intelligence.

 

Who Is Dr. Min Wanli?

Dr. Min Wanli (also known as Dr. Wanli Min) is the Chief Scientist of Machine Intelligence at Alibaba Cloud.

He holds a Ph.D in statistics from the University of Chicago, was a researcher at IBM’s T.J. Watson Research Center and a senior statistician at Google. He now oversees Alibaba Cloud’s artificial intelligence projects.

 

ET City Brain & The Tianchi Platform

ET is an Alibaba Cloud designation that refers to artificial intelligence services that “can be broadly applied to different areas in society”. It accomplishes this by leveraging the crowd intelligence capabilities of the Tianchi platform.

Dr. Min Wanli showcased how the Hangzhou City Brain master plan eases traffic congestion in the city, using an ambulance as an example. The Hangzhou City Brain will predict the traffic conditions for the next 30-60 minutes and determine the most efficient route for the ambulance.

It also synchronises the traffic signals so that they will turn green 10 seconds before the ambulance arrives, allowing it to pass without stopping. This not only cuts the ambulance’s arrival time, it also reduces risk of accidents.

 

The Malaysia City Brain

Dr. Min Wanli was here as part of the Alibaba Cloud delegation to announce their collaboration with the Malaysia Digital Economy Corporation (MDEC) to introduce the Malaysia City Brain.

[adrotate group=”2″]

In the first phase of implementation, the Malaysia City Brain will be used in Kuala Lumpur’s traffic management. It will begin with a base of 382 cameras, and input from 281 traffic light junctions – all located in central Kuala Lumpur.

Using cloud computing and big data processing capabilities, the Malaysia City Brain will be able to optimise the flow of vehicles and timing of traffic signals.

It will also be able to generate structured summaries of data like traffic volume, and speed according to lanes, which can be used to facilitate other tasks, including traffic accident detection.

Go Back To > Events | Home

 

Support Tech ARP!

If you like our work, you can help support our work by visiting our sponsors, participating in the Tech ARP Forums, or even donating to our fund. Any help you can render is greatly appreciated!