Blog Details

Facebook’s Plan To Design Its Own AI Chips Has Set Alarm Bells Ringing For Qualcomm, Nvidia And Intel

[Ed. note: see Sept. 2018 publication, Webscale Network Operators: 3Q18 Market Landscape]

Rumors about Facebook’s likely entry in the hardware segment were put to rest after the company posted job openings for chip designers. Facebook was looking for candidates specialized in architecting and designing ASIC and FPGA chips, to help build “custom solutions targeted at multiple verticals including AI/ML, compression, and video encoding.” Facebook has data center applications on the mind, such as live video content filtering, but the company may also be building chips to support its Oculus virtual reality headset and long-planned smart speaker.

Facebook’s chip plans are risky but may bring increased control over supply chain

Facebook’s recent move to make its own artificial intelligence (AI) chips and get a foothold in the hardware segment is a step in the right direction, as it looks to reduce dependence on chip manufacturers (Intel, Nvidia and Qualcomm) while putting a lid on its costs. However, designing a chip is by no means a simple task and not a core competency at Facebook. Matching the performance and efficiency of Intel, Nvidia and others will be a challenge.

Why would Facebook take the risk? It’s loaded with cash ($41.7B in Dec 2017) and used to making high-stakes tech investments. But the chip market is competitive, and Facebook has substantial buying power – it could certainly rely on the open market. Time to market may improve with self-design, for sure. However, another benefit may be more persuasive: Facebook gets greater control over intellectual property rights and information flow. It likely has a few surprises in store.

Big technology investments needed to support social networking

Facebook’s growth has been driven by acquiring and strengthening complementary services to its social networking business, such as WhatsApp and Instagram. Supporting this growth enticed Facebook to build a huge core network.

Like other webscale providers, Facebook works with contract manufacturers to build custom servers and other gear for their massive data centers. Facebook has played an important industry role in this regard, serving as an early sponsor for both the Open Compute Project (OCP) and Telecom Infrastructure Projects (TCP). Now Facebook is testing the waters in consumer electronics markets, initially with its Oculus virtual reality headsets, a smart speaker to be launched in 2018, and complementary AI software. These efforts have contributed to both high capital spending and R&D expenses at Facebook (Figure 1).

Facebook believes that creating its own custom designed chips will result in better integration of hardware and software, and give it tighter control over the development of the product. One factor behind Facebook’s chip push is an interest in running AI algorithms in-house, to avoid sharing with third-party vendors like Intel or Qualcomm. Not every webscale company can do this, but Facebook is positioned better than most due to its deep capex budget & its pioneering work at the OCP, TIP, and other groups.

Working in Facebook’s favor is its recent partnership with Intel to manufacture its own AI processor last year.

Impact of Facebook’s entry into the semiconductor space on the big chipmakers

In the past two years, there has been growing tension between the tech players building cloud networks and the vendors they rely on, mainly Intel, Qualcomm, and Nvidia. Historically the largest cloud builders, “webscale network operators” (WNOs) in our terminology, have heavily relied on Intel’s microprocessors and Nvidia’s GPUs to power their data centers (Figure 2).

However, Facebook is not the only WNO to look at building its own chip. Many webscale providers are starting to look in-house to build custom AI chips to reduce costs and improve on efficiency. For instance, Google developed an AI chip, Cloud Tensor Processing Unit (TPU), two years back, to boost its AI workloads. Google released a latest version this month, indicating that Nvidia’s dominance as a supplier of AI chips could soon be in jeopardy. Similarly, Apple plans to build its own chips for its Mac desktops by 2020, thus reducing its dependence on Intel. Amazon is building its own custom hardware to improve its Alexa enabled devices. Microsoft has launched a new cloud service for image-recognition projects powered by its FPGA technology, codenamed “Project Brainwave”.  This will rely on Intel Stratix 10 chips and support a neural network based on the ResNet-50 architecture. Microsoft claims that this new technology will be capable of handling AI tasks rapidly enough to be used for real-time jobs and at a reduced cost in comparison to the graphics chips (e.g. NVIDIA) used in machine learning tasks.

As webscale tech players build more of their own chips, traditional chip developers are getting nervous.

One company affected in a big way by Facebook’s move is Qualcomm. Facebook’s new chips may be used to power its VR headset, Oculus Go, which currently runs on a Qualcomm Snapdragon 821 chip. This could be a huge blow for Qualcomm. And it comes at a time when Qualcomm is already struggling after its legal battle with Apple, an attempted acquisition from Broadcom, and a still-pending merger with NXP.

The chip vendor’s fears are not just theoretical. Webscale players have already had an impact on the supply chain, hurting server vendors like IBM and HPE in past years by going to white box/contract manufacturing. Now they’re big enough to design their own chips. That cuts out the middleman, avoids having to share secret IP, maybe speeds time to market, and may result in some proprietary advances.

Facebook will have to win back faith amidst data privacy scandal

While Facebook engineers will continue to find ways to make their network cheaper and smarter, the company faces more complex challenges in the area of data privacy & public perceptions.

Facebook’s privacy practices have come under global scrutiny in recent months, due to the recent Cambridge Analytica and Android call data scandals, not helped by a photo tagging-related lawsuit. As a consumer-facing brand with plans to build its own IoT hardware, a lot is at stake. The company needs to build trust and improve transparency. It cannot do this while also maximizing ad revenue growth. Not all Facebook executives seem willing to accept this.

Amid all the public outcry, the launch of Facebook’s smart speaker has unsurprisingly been delayed. With recent news that Amazon’s Alexa has some interesting privacy-related glitches, Facebook’s decision is probably best. Now seems like a good time to focus on the basics.

Blog Details

A Telecom Analyst’s Take On CES 2018

For most of my career, I’ve been focused on the telecom industry and its components. I’ve been to dozens of telecom-focused conferences & exhibitions, in Asia and the Americas. I had never been to a consumer-focused show, though. In order to learn a bit (and check out some cool new devices), I spent a few days at CES in Las Vegas last week.

It was as chaotic as promised, but also a geek’s paradise. Loads of new tech was shown off in AI, IoT and smart cars. Telcos had a limited presence, but another type of network operator – those building webscale networks – was well represented.

Webscale at CES

Of the biggest companies building webscale networks, most had some sort of CES presence.

As covered widely elsewhere, Google’s Assistant and Amazon’s Alexa were hard to hide from, and overpowered Microsoft Cortana’s limited presence. Apple’s late 2017 decision to postpone the launch of its HomePod (powered by Siri) prevented it from making any kind of CES splash. The absence of Apple in the market, and Cortana’s failure to withstand the competition, left the turf wide open for Google to take on Amazon.

Monorail sponsorship paying off for Google

Chinese providers Baidu and Alibaba were also standouts; both sent impressive speakers and invested heavily in booth space. Baidu’s CES presence had self-driving as the centerpiece. The company formally announced its “Apollo 2.0” platform, in collaboration with 90 partners. Its booth showcased partners’ applications of Apollo in various mobility scenarios, including passenger vehicles, public buses and shared transport services. At the event’s “Mobile Innovation” keynote session, Baidu’s COO Qi Lu expanded on the driving focus, saying the company is “scaling everything around cars”. One positive for vendors: Lu argued that the transition to 5G should accelerate because of advances in AI – and the attendant need for more speed, security, and mobility.

For its part, Alibaba positioned itself well as an industry matchmaker at CES. It sponsored dozens of tiny suppliers in its “sourcing” tent – all of which use the Alibaba platform to serve customers. These suppliers sold every type of electronic under the sun. Some don’t even have products; Alibaba GM Kuo Zhang explained that he encouraged Chinese companies with “incomplete ideas” to come to CES, to meet people. On the buyer side, Alibaba explained in a breakout session how it aims to “de-risk” transactions by providing services like virtual reality factory inspection. That not only drives commerce on Alibaba.com, it also generates lots of traffic for the Alibaba Cloud to manage.

To provide some context, Figure 1 illustrates network-related spending for the top 8 webscale network operators, in 2016. As shown, Baidu & Alibaba are among the smaller companies, but both are growing quickly.

Figure 1

Source: MTN Consulting, LLC

Robots? Be patient. Drones? Watch your head.

CES had hundreds of companies demo’ing robotics of various flavors. This is far from my usual focus, but intriguing. Luckily, I got an hour before CES opened to tour the robotics section. Lots of neat toys, but my impression is the space is very early stage. In a conference session, a speaker noted that the closest thing to a mass market consumer robotic device so far is a vacuum cleaner, iRobot’s Roomba. While this is now being equipped with WiFi connectivity, app control, and dead zone detectors, (because, why not), it’s a simple product yet still only has sold 20 million units since its 2002 release; Apple sold over 200 million iPhones in its last fiscal year.

Beyond household appliances, there is a lot of innovation around sports & games. For instance, one exhibitor demo’d a robotic ping-pong player; cool, but rudimentary so far, and hard to see a mass market application. They will come, though.

Omron’s ping-pong playing robot

As for drones, also beyond my usual telecom focus, they were all over the place. Up, down, and in your face. The range of applications (agricultural monitoring, vaccine delivery), form factors, and swarming capability was impressive. But as with many devices, drones come along with privacy and security issues. China’s drone industry is proliferating rapidly, and aiming for US growth. That could raise some national security implications. With news last week that Huawei & ZTE are facing political opposition in the US again, watch this space.

IoT devices

For anyone skeptical of the Internet of Things, CES did not disappoint. Loads of ideas seemed to have little practical use, or were overly complex. The Daily Beast’s recap put it well: CES Was Full of Useless Robots and Machines That Don’t Work. The popular Internetofshit Twitter feed suggested CES should just be renamed IOS.

But this is too easy a critique. The market is young, and the barriers to entry are low – naturally lots of inane ideas get floated. And let’s not forget that major innovations often have unexpected sources, or look silly at the time. One exhibitor, Petrics, was presenting a smart dog bed last week, equipped with sensors to monitor weight & activity. I laughed at first, but the only pet I own is a desert tortoise. Not much of a commitment. Dog-owners, though, spend hundreds of dollars per year on food and medical care. In the US, it works out to about 1% of spending for the “average” household; pet-owning households spend more. Dogs are often integral members of their owners’ family, so naturally health is important. IoT for pets could go somewhere.

Petrics’ Smart Pet Bed

Who will benefit from this sort of thing? Telcos clearly want a piece of any IoT action, and have home networking & monitoring solutions to target this, on top of connectivity. The actual revenues from these sorts of consumer-focused services are largely speculative though.

Among the many reasons for this: interoperability in the IoT space is not well developed. That was made clear at the CES session on “Connected Ecosystems”. These devices not only have to work on their own – which they often don’t – but also interoperate with other devices. T-Mobile VP for IoT and M2M, Balaji Sridharan, noted that there is “huge value in two or more IoT systems talking to each other,” as interoperability is mostly ad-hoc right now. Zigbee’s President, Tobin Richardson, says it aims to help create a “frictionless environment” for connecting devices, but admitted this is extraordinarily complex in practice. Even for a relatively simple use case, lighting, just defining “on” and “off” in a standard is not straightforward. And Zigbee is not the only standards/certification body to consider; the Open Connectivity Foundation is also important. There are also a wide range of other standards, certification, energy usage, and other bodies relevant to specific types of equipment in the home (televisions, speakers, etc.) and for general safety (e.g. NSF International).

Beyond ease of use and interoperability, security & privacy is crucial in the home. Sridhar Kumaraswamy, who oversees Home Systems for Philips Lighting, noted that devices & home networks must not only be secure when installed, but easy to keep updated since consumers tend to be busy and not technically sophisticated. Currently, IoT is very much a “caveat emptor” environment for consumers. On that note, Amazon Web Services’ GM for IoT Analytics & Applications, Sarah Cooper, noted that AWS assumes it cannot secure every device on the network, so it focuses on monitoring “behavior and deviations.” That’s increasingly the approach taken by cloud-based providers.

The rise of “AI first” companies

Companies old and new see the benefits of artificial intelligence (AI)-based tools; that was an important theme at CES. Some are going further, and putting AI at the center of their messaging. Baidu and Google speakers both emphasized last week that they were “AI first” companies, or had AI at the center of their strategies. IBM is getting there, as it develops Watson and leverages recent acquisitions. These three companies are between 17 (Baidu) and 106 years old (IBM), though. While all three are positioned well now (along with several others), is this market likely to be kind to incumbents? Over three years, maybe, but 10 or 15? The shift to autonomous autos alone is likely to create some new industry giants we haven’t yet heard of.

As awe-inspiring (and frightening) as some AI innovations are – especially when combined with robotics – it’s early. Humans are still in control. The singularity isn’t here yet. Moreover, AI has limitations. Lacking a moral code is one; as Cisco’s VP for Worldwide Services Strategy & Innovation, Rajat Mishra, noted, “we cannot outsource morality to AI.” That puts the brakes on lots of things. Or should. IBM Watson’s CTO Robert High reminded the audience of another limit of today’s AI: “these things are not deterministic, so you should not apply to applications that require high levels of accuracy” such as financial statement auditing.

One takeaway from CES overall, in the AI arena: tech companies are not being frank about job loss questions. When this issue arises, the answer is often something like this: AI will improve the drudgery of jobs, and let employees focus on more meaningful tasks; some job loss may occur in the transition, but it will be the dull jobs that go away. There is plenty of truth to that. But it’s also true that companies are already investing in AI, and it’s often specifically in order to reduce headcount. The use of things like chatbots in call centers won’t decimate entire industries, but things will get worse. This is something we’re watching closely in the telco arena. We expect telcos to be get more aggressive about cutting staff count in the next 2 years, and AI tools are one way to get there.

Blog Details

Microsoft & Google pressure Amazon’s prime spot in the cloud; Apple may be prepping for entry

After a decade of dominance in the public cloud market, Amazon’s top spot came under immense strain in 2017. Years of investment in networks & cloud services began to pay off for Microsoft’s Azure and the Google Cloud Platform (GCP). Apple is also making noises in the cloud.

Tides shifting in 2017

At year-end 2016, Amazon’s Amazon Web Services (AWS) was the public cloud’s market leader, as it had been for many years. Per the Cloud Security Alliance, in 2016, AWS had a 42% share of the public cloud application installed base, Azure had 29%, and GCP had 3% (as did IBM’s SoftLayer).

Capex spend has been strong at Amazon’s rivals for many years, though, and it appeared to pay off in 2017.

As shown in the figure below, capex has grown at all three since 2012, but faster at Google and Microsoft. Each of these “webscale network operators” (WNOs) spend capex on items unrelated to the cloud, for instance Amazon’s fulfillment centers, or Microsoft’s retail outlets. But the big driver in the last few years has been cloud capex, concentrated around construction (or expansion, or retrofitting) of data centers, and supporting infrastructure such as data center interconnect.

MTN Consulting - cloud capex WNO

All this cloud investment has created an intense rivalry, with the new entrants pushing hard on Amazon’s top spot. Large and medium enterprise customers (such as Target, Apple, Dropbox, and Spotify) are now looking at alternatives, including a partial shift from Amazon to other leading cloud service providers. Amazon’s loss of such enterprises from the AWS fold has hurt operating margins, not just topline growth.

Microsoft’s “not so soft” approach appears to be hurting Amazon’s AWS margins

Amazon’s AWS unit has continued to grow fast in 2017, but at declining rates: year-over-year (YoY) revenue growth for AWS was 42% in 3Q17, down substantially from 3Q16. By contrast, Microsoft’s Azure revenues have grown at an average of more than 90% in recent quarters (chart, below).

MTN Consulting AWS-Azure rev grate

More important, after margin declines in 2016, Azure saw improvement in the last two quarters. As the chart below shows, Microsoft’s “Intelligent Cloud” (Azure) margins have improved YoY for the last two quarters, while AWS margins did the opposite. The two companies’ margins converged somewhat in both 3Q16 and 3Q17, but Microsoft’s overall level is safely higher. 

MTN Consulting AWS-Azure margins

To support Azure’s growth, Microsoft has invested on multiple fronts, including acquisitions. For instance, the company recently acquired Cycle Computing, a startup software developer that allows businesses to run apps in the cloud, a lucrative business for cloud vendors. As Cycle Computing has been a long-time partner with AWS and Google, Microsoft gets some new customers out of this acquisition: the existing Cycle Computing customers on AWS and Google Cloud will be asked to migrate to Azure, along with the future customers.

Microsoft has also invested heavily in network capex, partnering with such vendor suppliers as ADVA (100G optical for DCI); Cisco (Cisco Cloud Services Router 1000V); Ericsson (IoT accelerator); Huawei (a jointly engineered server product for hybrid cloud apps on the Azure Stack); Mellanox (40G Ethernet switches); Qualcomm (evaluating the new Qualcomm Centriq 2400 processor for cloud applications); and, many others.

Cloud is one of Google’s three big bets

Google Cloud Platform, the tech giant’s cloud division, currently lags far behind AWS but is trying to catch up. The renewed focus on cloud is a result of Google exploring growth outside its core advertising business. This continues to grow nicely, but remains highly vulnerable to economic headwinds.

Google’s CEO Sundar Pichai says “Cloud” is among the top three bets of the firm going forward. Unlike Microsoft, Google does not break out revenues (or report margins) for its cloud business separately. However, we know that GCP is a big part the company’s growing “Google Other” segment, which was 12.3% of total revenues in 3Q17 (3Q16: 10.8%). Google’s CFO Ruth Porat confirms that GCP is a main driver for growth in this segment. One metric of GCP’s growth is the number of big (>$0.5M) cloud deals signed per quarter; the total in 2Q17 was 3x the total for 2Q16.

To support this growth, cloud-specific investments have increased significantly in 2017; overall capex was $3.5B in 3Q17, up 39% YoY, benefiting server manufacturing partners Inventec & Quanta Computer. Cloud opex is rising as well, due to new cloud technical & sales staff hires.

Google racks PRY_20

Google’s push into the cloud market is only a few quarters old

The moderate gains made so far by Google in the cloud market are impressive, considering they just started in 4Q15, with the appointment of VMware co-founder Diane Greene to head its cloud business.

Prior to Greene’s appointment, Google was mostly perceived as the Internet search and advertising giant, which struggled to market cloud solutions to enterprises. The perception has since changed a bit, with GCP’s aggressive pricing strategy and incremental market gains. The GCP got a big boost in September 2017 with a win at Salesforce. Earlier this month, Google made another important hire: Diane Bryant, Intel’s former datacenter unit head, is becoming the COO of Google Cloud.

With this impressive team, Google is now looking to outperform AWS by 2022. Five years is ambitious, but not impossible. To succeed, Google is looking to position itself as a cloud solutions provider for AI- and Big Data-based applications, as these two technologies are considered as next big key adopters to cloud. Google is starting to reap some results from this new positioning. For instance, it recently struck a deal with Zebra Medical Vision to host its AI algorithms on Google’s cloud.

However, Google has to do much more than enticing big-ticket enterprise customers to switch. For rapid growth, and to support an AWS-like breadth of offerings, Google would need a sizable acquisition. Google’s biggest acquisition so far has been Motorola for US$12.5 billion. It would be looking to make a similar-sized acquisition in the medium term to help catch up to Amazon. Per the rumor mill, Salesforce and Workday are options, among many others.

 In the long run, Apple’s project “Pie” could eat into Amazon’s “share of pie”

While Amazon is focused on Microsoft and Google in the short to medium run, Apple may be secretly beefing up its own cloud capabilities to battle Amazon in the long run.

Currently, Apple’s role in the cloud has been mostly in the SaaS space through its iCloud service. However, a number of indicators point to Apple pursuing its own cloud computing strategy beyond SaaS; for example:

  1. Secret restructuring of its cloud computing operations under a project code named “Pie”: This includes moving the infrastructure for Siri, iTunes, Apple Music and Apple News onto a single proprietary cloud platform called “Pie”.
  2. Reduced reliance on other cloud operators to run its iCloud and other services: Apple is having issues relying on other cloud providers, as slow networks and outages disrupt Apple’s services. In late 2015, Apple started exploring how to build its own cloud infrastructure and end dependence on other cloud players completely, through “Project McQueen”.
  3. Increased investments around data centers: in 2017, the company announced two massive data centers in Iowa and Nevada, with construction costs of US$1.3B and US$1.0B respectively.

All the above specifics clearly suggest Apple aims to make a foray into the cloud market. But by no means guarantee it. Apple does not enjoy being predictable.

Amazon fights back

Amazon is not sitting back in face of these threats. The company is adopting a three-pronged strategy of “Innovate-Invest-Collaborate” in the cloud.

As the cloud pioneer, AWS continues to “Innovate”: expanding cloud platform functionalities from 280 new features in 2013 to 1,000 new features in 2016; launching a joint innovation center in Qingdao, China in 2017; rolling out a the AWS Snowmobile, Snowmobile.6824c527b221bfcd0fc284a04576b23d0d5edc1fwhich is a physical data transfer service using a 45-foot long container on a truck. That’s for transport to and/or between AWS data centers. Amazon also continues to “Invest” in (or acquire) cloud-related companies, including cyber security company Harvest.ai in January 2017, Thinkbox Software in March 2017, and GameSparks in July 2017.

The “collaborate” aspect of Amazon’s strategy involves collaboration with rivals where Amazon is weak. For instance, Amazon announced a surprise partnership with Microsoft in October 2017, to launch a free software tool for developers, Gluon, which allows them to build AI and cognitive systems. The alliance is seen as countering Google’s TensorFlow tool, which is already popular among developers. In 1Q17, Amazon teamed up with rival VMware to develop software to help companies move on-premises applications to the public cloud. These moves are significant for Amazon, as it looks to counter its peers in specific product segments in order to maintain its #1 position and lift margins.

For more information about MTN Consulting’s coverage of webscale network operators, please email us.

(Photo sources: Google & AWS)