About 20% of telecom’s 100 or so key vendors have now reported second quarter 2019 (2Q19) results. From these early vendor results, there are modest signs of a ramp-up in 5G-related spending.
Preliminary totals indicate growth of +1.9% YoY in vendor revenues to telecom operators (or telcos). Revenues for all vendors dropped last quarter, by 0.6% YoY, so this would be a slight trend reversal.
Ericsson only big NEP to report so far
Of the vendors reporting so far, the only large Network Equipment Provider (NEP) is Ericsson. (We also track IT services providers, and fiber/cabling vendors selling to telcos). Ericsson is also by far the largest to report so far, accounting for over 50% of reported revenues.
Per MTN Consulting estimates, Ericsson’s telco sales grew 2.1% YoY (on a USD basis), near the market average of 1.9% to date (see figure, below). That’s the fastest growth seen by Ericsson in several years, but it appears to have come at a price. Ericsson notes a negative margin impact from its push for “strategic contracts” in the Networks division.
There is a broad range of growth rates around the Ericsson-driven average. Vendors are finding growth in different aspects of the market, including high-capacity switches & open networking (Accton), high-speed test equipment (EXFO), 5G-related services & software (Infosys, TCS, Wipro), FTTx (Adtran, Nexans), and digital transformation consulting (Accenture). Some of these vendors sell to multiple segments, some are more specialized in the telecom vertical.
Source: MTN Consulting estimates of vendor sales to telcos (adjusted for M&A, US$ basis)
Some vendors also saw revenue dips in the telco segment, per our estimates; that includes Oracle and IBM, most importantly. These two vendors sell a range of software and services to telcos, as well as some network equipment, but are facing new competition. For example just last week Microsoft signed a large cloud deal with AT&T, a multiyear collaboration to help lower the company’s network and IT costs, moving more apps to the public cloud. At the same time, AT&T also expanded an existing cloud partnership with IBM. Both Oracle and IBM have annual sales to telcos in the $2-3B range, so don’t count them out.
The revenue drop shown above for TE Connectivity is estimated: SubCom is now part of Cerberus Capital, and does not report. However, SubCom’s pipeline was weak at the time of acquisition and deal integration usually causes a slowdown. Parts of the submarine market are picking up though, due to webscale investment and much-needed gap-filling in the Middle East & Africa. Nexans appears to be a beneficiary. Corning, Prysmian and other key fiber suppliers have not yet reported.
Growth trajectory remains modest
The figure below compares YoY growth rates for the sample with the market, i.e. the sum of all companies in MTN Consulting’s telecom vendor share coverage database.
Source: MTN Consulting
When including all vendors (black line, above), revenues have largely been flat over the last several quarters. The sample of companies reporting appears broadly similar. However, on the demand side, guidance from telcos on expected spending levels (capex and network opex) was quite conservative for 2Q19. The final growth rate for 2Q19 vendor revenues in the telco vertical will likely be below +2%.
To reiterate findings from our latest (1Q19) telco sector Market Review:
Telco profit margins remain tight, nothing new for the telecom industry. Operators are getting more concerned about debt, though. The net debt (debt minus cash) of the global telco sector was roughly half of revenues in 2018, after having been in the 30-40% range of revenues at the cusp of the LTE buildout cycle. Few telcos have room in their budgets for a 5G capex splurge.
Telco network investments continued a declining trend, as capex touched $70B in 1Q19, down almost 2.5% YoY. The weak 1Q19 result and continued supply side uncertainty does not bode well for 2019. The slowdown could be due to operator caution about market demand. Yet competitive realities will require operators to spend big on 5G and fiber in 2019-20. The market’s average capital intensity will exceed 17% by the end of this year.
We expect to publish further commentary on the market after Nokia and Samsung report next week.
In the last few years the demands from webscale network operators (WNOs, Figure 1) on transmission network architectures have changed considerably. From pure raw capacity requirements and lower costs, webscale players now prioritize highly scalable and advanced point to point bandwidth bundling interface technologies.
Figure 1: List of Webscale Network Operators (WNOs)
Source: MTN Consulting, LLC
Webscale operators’ field of expertise is the data center, and most planned at least initially to rent capacity as leased lines from telcos (or telecommunications network operators, aka TNOs). However, many TNOs did not have the end to end transmission networks able to support webscale needs in terms of capacity, latency, cost objectives. Further, telco networks were not flexible enough to follow rapidly WNOs’ needs for modifications, additions, and changes of the services they needed.
Hence several years ago, WNOs themselves decided to build their own backbone and regional transmission networks, sometimes linking continents. Undersea, the WNOs either leased capacity from existing submarine consortia systems or started to build submarine cables for their own dedicated use. The largest WNOs, such as Microsoft, Facebook and Alphabet, have increasingly favored the latter (self-build) approach. With these initiatives, WNOs seek to have full control of the transmission network, and adequate time to market for their needs.
As webscale players have built out their networks, they have become more influential across the industry. Their buying power alone is a major reason; Figure 2 shows how webscale operator capex has grown dramatically since 2012, while telco capex has stagnated.
Figure 2: Capex – telco vs webscale (US$B)
Source: MTN Consulting, LLC
At the same time, the telco market is far larger, and the largest integrated telcos spend well over 10% of capex on their transport networks. These telcos are heavily investing in the transformation of their transport networks, and supporting 5G is a central goal.
In the past, mobile services have been sold on the back of convenience of use. With very little considerations beyond coverage and without a capacity objective, there was never a firm commitment to service quality. 5G is probably the first access network with services subject to a wide variety of SLAs, from best effort to non-congested, and very low latency services with limits as low as 1ms. 5G will help operators to move from best effort services for all, to a tiered service level agreement (SLA)-based portfolio. Telcos hope this will help them to be more profitable, at least for the more sophisticated services.
Network slicing poised to play important role
The big change in direction in the strategy of transmission networks is that the planning, design, engineering and operations of a telecom operator will soon be subject to much tighter contracts and commitments.
For years, wild overbooking levels have been the norm, especially for mobile services, and networks were in most cases engineered for coverage alone. This won’t be possible for the next generation of services, which will require more than a 10x increase in bandwidth, and 10x less latency than the current generation.
In addition, webscale operators and large enterprises have demanding network KPI requirements. To serve this market, telcos must develop their transmission network end to end with enough flexibility to satisfy the capacity growth needs and resiliency requirements of these customers.
TNOs and WNOs both accept that the demanding requirements on bandwidth, latency and operational scalability to ensure short time to market for 5G services cannot be supported with existing network architectures.
A potential solution is “network slicing”, which starts with adding more TDM capabilities in the data plane to be able to provide a hard separation in the way services with different KPIs use network resources. This separation is orchestrated by an SDN centralized control and management plane.
Network slicing brings improvements to traffic engineering, with clear KPIs for bandwidth, latency and packet congestion. That helps to support all types of services over the same network infrastructure. Low priority services such as web browsing are effectively separated from network resources dedicated to services with demanding SLAs such as low latency leased lines or 5G inter-vehicle communications.
Operators pursuing FlexO technology to help cope with looming Shannon limit
Historically the main requirements telcos have standardized for transmission network architectures and platforms are high resiliency, powerful operations administration maintenance features, multiservice support, and backwards compatibility with legacy platforms.
This makes a lot of sense as most of the costs of running the network are operational in nature, such as repairs and maintenance. Further, multiservice capabilities can facilitate the migration of legacy services to newer platform. This reduces the need to support overlaid networks, and also avoids the cost of capacity expansions on older platforms at or near their end-of-life (EOL) dates.
In recent years, technological developments have pushed transmission networks towards the limit in the bandwidth per distance product, or the “Shannon limit”. The transmission technology is starting to hit the limits of the fiber medium.
One way to cope with this comes from the ITU, with its Flexible OTN, or FlexO, standard (G.709.1/Y.1331.1). FlexO allows client OTN handoffs above 100Gbps by defining an “OTUCn” modular structure: “an aggregate OTUCn (n ≥ 1) can be transferred using bonded FlexO short-reach interfaces as lower bandwidth elements.” FlexO also supports standard 100GbE optical modules.
FlexO has led telcos to consider how to fully exploit the flexibility of coherent transmission systems, allowing very high capacity transmission on non-regenerated short links, say 400Gbps links over 300Km distances, and lower capacity transmission over longer links, for example 100Gbps over 1500Km distances (figures for illustration only).
FlexO can bundle a number of lower rates at the TDM level to serve a higher capacity service for very long distances. For instance, by using inverse multiplexing or bundling a 400G service interface, capacity could be carried over four 100G links over (for instance) 1500kms without regeneration.
True to their backwards compatible requirements, telcos have made sure that FlexO supports 100G transmission requirements, and is an extension of existing OTN standards. This should simplify the roll out of FlexO on existing platforms.
FlexE to improve utilization, end-end manageability and router-transport connectivity
Operators – both telco & webscale – have also been exploring breakthroughs in the interfaces between transmission systems and servers and routers.
Aligned with FlexO, the Optical Internetworking Forum’s Flexible Ethernet (FlexE) supports similar schemes of bundling and multiplexing of interfaces between routers and transmission systems. FlexE offers a way to transport a range of Ethernet MAC rates whether or not they correspond to existing physical (Ethernet PHY) rates. Network utilization should improve, as should end-end manageability. One key element of FlexE was that Ethernet would grow within a TDM frame. This may pave the way to network slicing through the use of hard boundaries between tranches of services with different SLAs.
Most webscale operators lack an access link to the end user, making them rely heavily on telcos. And smaller webscale players like Netflix rent their clouds from other providers. Maintaining control of the user experience is an uphill battle. FlexO and FlexE help achieve this, in theory. On the UNI side, a WNO transmission network would now use FlexE interfaces with data platforms and servers and storage. On the NNI side, towards the fiber and other transmission systems, the WNO would use FlexO interfaces and standards.
Transmission interoperability improving due in part to the webscale push
Interoperability is something that transport engineers always wish for but never achieve due to network management interfaces’ lack of interoperability. Further, with coherent transmission, there is a problem with transmission interface incompatibility between vendors, each of whom can be more interested in higher performance and features differentiation than simplicity.
The telco response to the interoperability challenge has generally been to achieve subnetwork level interoperability rather than network element interoperability outright.
Things will change, though, as FlexO could be called the first optical standard that thrives on multivendor equipment operations.
Furthermore, webscale operators have designed simple transmission platforms and aimed to use cheap components already available from larger industries. Examples include the use of Ethernet interfaces components at 25G and 50G that were originally proposed for intra data center connectivity and rack cablings between servers and top rack unit switches. These will also be used in 5G base stations and mobile cloud engine platforms that require a transmission network to interconnect.
Conclusions
There is a growing alignment in the requirements for transmission network architectures across telecommunications and webscale network operators. They both need more flexible ways to grow their networks and manage them on an end to end basis. They want to benefit from low cost, open source components and procurement, but adapt technology to suit their customer base. They need to be able to support different classes of service and traffic. Even when providing free services, operators need to deliver a high quality of experience in order to monetize.
5G transport and data center interconnectivity services pose such a challenge to both TNOs and WNOs that work-arounds will not make up for limitations in either the data centers nor the network. For many operators, building a transmission network that supports network slicing principles will require a fresh start and new investments.
–
Source of cover image: CommScope.
[Note: a condensed version of this article first appeared at Telecomasia.net.]
AT&T and Verizon both reported first quarter 2019 (1Q19) earnings this week. Analysts expecting a capex bump from 5G were disappointed. Capital expenditures (capex) fell at both companies on a year-over-year (YoY) basis, down 14% at AT&T and 6% at Verizon. Capital intensity at AT&T fell to what is probably an all-time low in 1Q19: capex is now 11.2% of revenues, on a 12-month annualized basis. Verizon’s 12.5% annualized capital intensity is also near its all-time low. Both telcos continue to cut headcount: their combined employees fell by over 11,000 in just three months, from December 2018 to March 2019.
Our view
AT&T and Verizon continue to hype 5G and aim to position themselves as 5G leaders – even before the network capabilities and services are really there. This is not a new phenomenon, or unique to the US.
While they want brand leadership, that doesn’t mean they want to spend as much on 5G as they can. In this environment, they prefer to spend as little as needed. Top-line revenue growth has been weak, and operating margins flat. Both telcos are spending big on acquisitions and other investments as they expand into cloud and media markets. They’re also paying down their debt. For example, AT&T’s year-end 2019 goal is net debt of $150B, from $169B currently – that requires selling assets, including office properties and a Hulu investment. Actual capex outlays on 5G-capable networks will be incurred gradually, as they can be justified. Marketing is much easier and faster to scale than network construction.
Moreover, some aspects of network construction are getting cheaper & easier. Open networking/open source solutions have matured dramatically over the last few years. Both telcos are using solutions from this community in some aspect of their 5G deployments, even if not the RAN. AT&T is more vocal; it has plans to deploy up to 60,000 “white box” cell site gateway routers across its mobile network (eventually). Earlier this month, AT&T demonstrated the UfiSpace white box (running Vyatta’s OS) at the Open Networking Summit in San Jose. Verizon is also incorporating more open source projects into its network. At both companies, the direct impact of this is small now, but white boxes as a share of telco capex will grow over time.
Even as both telcos spend cautiously on 5G, they are cutting headcount. AT&T’s total employees fell 2.2% in three months, to 262.3K. Verizon’s fall was steeper, down 3.5% from December 2018 to 139.4K employees.
Source: MTN Consulting, LLC
When telcos lay off employees – including voluntary retirement schemes – some cuts are related to mergers and “redundancies”, but most are part of a natural evolution. Operators are always in search of scale economies. Networks are more automated now, requiring fewer people to run than in the past. When operators converge business lines, fewer salespeople are needed. Both AT&T and Verizon have, like most telcos, been shrinking their workforce for many years.
Even with these cuts, they both spend much more on their staff than on capex (figure, above). This gap has remained wide for several years. Their capital spending has declined to record lows, and can’t sink much further as a percent of revenues. In order to fund a 5G uptick in capex, while also keeping debt in check, both operators will continue to slim their workforce.
Network disaggregation is one of those topics that is hard to build an audience around. The appeal of it mostly is on the cost side. It’s not short and enticing like “5G”. And it means different things to different people, even within the same operators’ network department. Yet OFC sessions made clear how important this concept is becoming for operators, both telcos and webscale.
Post-OFC news reinforced this, as Nvidia announced on March 11 that it would pay $6.9B for Mellanox Technologies. Mellanox has been an advocate of open networking for years in forums like the OCP and ONF, pitching its portfolio as an “Open Ethernet approach to network disaggregation.” For anyone wondering if this approach had market appeal, the Nvidia deal may have tipped the scales.
AT&T, NTT among the big telcos making moves towards disaggregation
In the optical networks space, disaggregation generally refers to open line systems, where systems and transponders are decoupled. That allows for faster upgrades of transponders, and avoids vendor lock-in. AT&T is on board here, as Scott Mountford confirmed at Monday’s “Open Platform Summit”, saying “we’ve been pretty vocal these last few years about open optical networks”. That includes founding support for the Open ROADM MSA, which was featured in a demo at OFC involving AT&T, Orange, Fujitu, Ciena, and the University of Texas, Dallas.
More broadly, large operators and select vendors have been trying to promote a “white box ecosystem” where hardware can be decoupled (or disaggregated) from software. AT&T made a splash in December when it announced it would deploy white box routers at up to 60,000 towers over the next few years. The company will release as open source the software it is writing for the routers. A large operator like AT&T can make this early commitment, but most others are more cautious. At a 5G session on Monday, AT&T Kent McCammon noted that standards bodies like the ONF, Open Compute, and Linux Foundation are important because in order “to reduce costs we need to simplify operator requirements around commonalities”.
For Japan’s NTT, lowering network opex is a central goal of the white box shift. Akira Hirano from NTT discussed how white boxes can help operators lower opex through “zero touch functions”. The company cited its use of the Cassini white box as a success, because it automates L3 network configuration, requiring only 1 command. However, the NTT speaker noted that the application is only for data center interconnect, and that “for sure” this is not in use in long haul networks yet.
AT&T also underscored the gradual nature of change in telco fiber networks. They have been built over decades, have a range of different attached network equipment, and are subject to a variety of depreciation rates. AT&T’s Mountford also noted that “operational systems need development” in this area, in order to actually manage decoupled network elements. That is something the webscale sector is able to attack more easily, given their relatively simple networks.
Google and Microsoft full steam ahead
The biggest webscale providers spend billions per year on their networks. Most have embraced open networking from the start. Microsoft’s Mark Filer stated at the Summit that “open and disaggregated networks are already powering Microsoft’s cloud”. In making this happen, he emphasized the importance of a set of software tools built internally, “Microsoft SDN”, which includes a topology engine, zero touch configuration tools, data collection tools, and alerts & correlations.
Similarly, Google’s Eric Breverman emphasized software in his talk on “Optical Zero Touch Networking”. The goal of ZTN, Breverman explained, is essentially to “keep people from actually touching the network”. Humans make mistakes, and they are too costly to keep hiring at the same pace as traffic. Automatic network configuration is important. Google says it now supports intent-driven networking on 50% of “Google’s Production Optical Network”. OpenConfig is important here, as it allows working across multiple vendors much easier than with TL1 and SNMP.
Telcos need software skills
It’s no surprise that telecom operators are eager to lower the cost of growing & operating their network. Open platforms have the potential to contribute, and not just in optics. Building the right software tools to manage these platforms is crucial, though, and webscale providers are further along than telcos. As an analyst, I have to wonder whether telcos need to reach deeper into their pockets for R&D budgeting. For AT&T, one of the biggest telco spenders, it spent just 0.7% of revenues on R&D in 2018, down from 1.3% in 2014. Webscale R&D spending averages out to about 10% of revenues, and it shows.
