Power grids around the world are facing similar challenges. One of the biggest is the rise in renewable energy generation of all kinds; solar and wind energy are great for the planet, but they are as unpredictable as the weather. Schemes designed to encourage consumers to put solar panels on roofs and use electric vehicles to store energy mean the grid is morphing from one-directional to bi-directional. And instead of demand prediction, utilities now need to predict both supply and demand in real time, at very fine levels of granularity.

“The ability to add AI into the mix and do real time analytics at the edge is going to be critical for increasing the amount of distributed energy resources that can come online,” Marc Spieler, global business development and technology executive for the energy sector at Nvidia told EE Times.

Spieler pointed out that great work is being done in wind, solar and electric vehicles, but if the grid doesn’t have the ability to support these applications, the effort is wasted.

Demand prediction draws on many complicated factors. Aside from the weather, real time prediction might include complex tasks such as predicting how many electric trucks will arrive at which filling station and require battery charging at what exact moment, for example.

“It’s going to come down to hour by hour, minute by minute type decisions,” he said. “And AI is the only thing that’s going to allow that to become efficient.”

Large-scale prediction
Utilities typically subscribe to detailed weather prediction services today, feeding this data into complex models to try to predict energy demand.

“The people doing this best are probably the financial services companies, the hedge funds that are buying and selling power,” Spieler said. “Those guys are making huge investments in AI and they’re capitalizing on the profit.”

However, Spieler said, utilities are upping their game.

“We are seeing a ramp up of data science in the utility,” he said. “Some of the [utilities] we’re working with are ramping up their data science communities. We’re starting to sell hardware DGX systems [Nvidia data center-class AI accelerators] into utilities for the first time.”

More details at:  https://www.eetimes.com/adding-intelligence-to-the-grid/?utm_source=newsletter&utm_campaign=link&utm_medium=EETimesDaily-20210722&oly_enc_id=9796A5331467C2S

Relational AI
Cynthia Breazeal, an associate professor of media arts and sciences at the Massachusetts Institute of Technology, focuses her research at the micro-level, exploring AI’s positive impacts on individuals in their everyday lives.

“I don’t think that AI will have fulfilled its full potential unless it can unlock human potential,” said Breazeal, one of several featured speakers at a recent academic conference about the intersection of technology and human psychology.

“Can AI help us become who we aspire to be? Can it help us live not just more productive but better, more fulfilling lives?”

There’s no question that personal robots and other AI systems are becoming a growing presence in our lives. But while Apple’s Siri, Google Home and Amazon’s Alexa can carry out certain tasks, they can’t truly support who we are, according to Breazeal.

“When we talk about human flourishing,” she said, “it’s not about brief encounters with AI. Positive emotions, meaning, a sense of achievement, relationships — all those things are extremely important. I call it ‘relational AI’ — AI that can understand us as people and treat us as people.”

Although Jibo ultimately failed…
Breazeal believes that we need to appreciate human social and emotional psychology and design the robots according to the principles of how people interact with one another. “A social robot,” she said, “should feel much more like you’re interacting with a someone rather than a something.”

Jibo, a robot the size of a small table lamp developed in Breazeal’s lab, offers a great example. Like other social robots, Jibo was designed to win us over not with its cognitive prowess but with its personality and sociability.

It was programmed to autonomously read out emails and make video calls, but, more importantly, to recognize faces, notice when someone enters the room, and turn its head to say hello or crack a joke.

Social Robots in Education
Breazeal has been studying the benefits of humanized engagement with AI in schools, using social robots as personalized learning companions.

We know that children learn best when actual humans give them individual attention; however, given overcrowded classrooms, a social robot could step in to supplement a human teacher, according to Breazeal.

More at: https://asianroboticsreview.com/home545.html

A new robotic neck brace from researchers at Columbia Engineering and their colleagues at Columbia's Department of Otolaryngology may help doctors analyze the impact of cancer treatments on the neck mobility of patients and guide their recovery.

Head and neck cancer was the seventh most common cancer worldwide in 2018, with 890,000 new cases and 450,000 deaths, accounting for 3% of all cancers and more than 1.5% of all cancer deaths in the United States. Such cancer can spread to lymph nodes in the neck, as well as other organs in the body. Surgically removing lymph nodes in the neck can help doctors investigate the risk of spread, but may result in pain and stiffness in the shoulders and neck for years afterward.

Identifying which patients may have issues with neck movement "can be difficult, as the findings are often subtle and challenging to quantify," said Scott Troob, assistant professor of otolaryngology - head and neck surgery and division chief of facial plastic and reconstructive surgery at Columbia University Irving Medical Center.

However, successfully targeting what difficulties they might have with mobility can help patients benefit from targeted physical therapy interventions, he explained.

The current techniques and tools that doctors have to judge the range of motion a patient may have lost in their neck and shoulders are somewhat crude, explained Sunil K. Agrawal, a professor of mechanical engineering and rehabilitative and regenerative medicine and director of the ROAR (Robotics and Rehabilitation) Laboratory at Columbia Engineering. They usually either provide unreliable measurements or require too much time and space to set up for use in routine clinical visits.

To develop a more reliable and portable tool to analyze neck mobility, Agrawal and his colleagues drew inspiration from a robotic neck brace they previously developed to analyze head and neck motions in patients with amyotrophic lateral sclerosis (ALS). In partnership with Troob's group, they have now designed a new wearable robotic neck brace. Their study appears July 12 in the journal Wearable Technologies.

The new brace was made using 3D-printed materials and inexpensive sensors. The easy-to-wear device was based on the head and neck movements of 10 healthy individuals.

"This is the first study of this kind where a wearable robotic neck brace has been designed to characterize the full head and neck range of motion," Agrawal said.

In the new study, the researchers used the prototype brace, along with electrical measurements of muscle activity, to compare the neck mobility of five cancer patients before and one month after surgical removal of neck lymph nodes. They found their device could precisely detect changes in patient neck movements during routine clinical visits.

"Use of the sensing neck brace allows a surgeon to screen patients postoperatively for movement difficulty, quantify their degree of impairment, and select patients for physical therapy and rehabilitation," Troob said.

In the future, the researchers hope to investigate larger groups of patients and use the neck brace to follow patients through physical therapy to develop evidence-based protocols for rehabilitation, Troob said. They also would like to develop similar braces for other surgical sites, such as the forearm, ankle, or knee, he added.

A simulation engine developed by USC and NVIDIA researchers predicts the forces acting on a knife as it cuts through common foods, such as fruit and vegetables.

Researchers from the USC’s Department of Computer Science and NVIDIA have unveiled a new simulator for robotic cutting that can accurately reproduce the forces acting on a knife as it slices through common food, such as fruit and vegetables. The system could also simulate cutting through human tissue, offering potential applications in surgical robotics. The paper was presented at the Robotics: Science and Systems (RSS) Conference 2021 on July 16, where it received the Best Student Paper Award.

In the past, researchers have had trouble creating intelligent robots that replicate cutting. One challenge, they’ve argued, is that no two objects are the same, and current robotic cutting systems struggle with variation. To overcome this, the team devised a unique approach to simulate cutting by introducing springs between the two halves of the object being cut, represented by a mesh. These springs are weakened over time in proportion to the force exerted by the knife on the mesh.

“What makes ours a special kind of simulator is that it is ‘differentiable,’ which means that it can help us automatically tune these simulation parameters from real-world measurements,” said lead author Eric Heiden, a PhD in computer science student at USC. “That’s important because closing this reality gap is a significant challenge for roboticists today. Without this, robots may never break out of simulation into the real world.”

To transfer skills from simulation to reality, the simulator must be able to model a real system. In one of the experiments, the researchers used a dataset of force profiles from a physical robot to produce highly accurate predictions of how the knife would move in real life. In addition to applications in the food processing industry, where robots could take over dangerous tasks like repetitive cutting, the simulator could improve force haptic feedback accuracy in surgical robots, helping to guide surgeons and prevent injury.

“Here, it is important to have an accurate model of the cutting process and to be able to realistically reproduce the forces acting on the cutting tool as different kinds of tissue are being cut,” said Heiden. “With our approach, we are able to automatically tune our simulator to match different types of material and achieve highly accurate simulations of the force profile.” In ongoing research, the team is applying the system to real-world robots.

Co-authors are Miles Macklin, Yashraj S Narang, Dieter Fox, Animesh Garg, Fabio Ramos, all of NVIDIA.

The high expectations of AI have triggered worldwide interest and concern, generating 400+ policy documents on responsible AI. Intense discussions over the ethical issues lay a helpful foundation, preparing researchers, managers, policy makers, and educators for constructive discussions that will lead to clear recommendations for building the reliable, safe, and trustworthy systems⁶ that will be commercial success. This Viewpoint focuses on four themes that lead to 15 recommendations for moving forward. The four themes combine AI thinking with human-centered User Experience Design (UXD).

Ethics and Design. Ethical discussions are a vital foundation, but raising the edifice of responsible AI requires design decisions to guide software engineering teams, business managers, industry leaders, and government policymakers. Ethical concerns are catalogued in the Berkman Klein Center report³ that offers ethical principles in eight categories: privacy, accountability, safety and security, transparency and explainability, fairness and non-discrimination, human control of technology, professional responsibility, and promotion of human values. These important ethical foundations can be strengthened with actionable design guidelines.

Autonomous Algorithms and Human Control. The recent CRA report² on "Assured Autonomy" and the IEEE's influential report⁴ on "Ethically Aligned Design" are strongly devoted to "Autonomous and Intelligent Systems." The reports emphasize machine autonomy, which becomes safer when human control can be exercised to prevent damage. I share the desire for autonomy by way of elegant and efficient algorithms, while adding well-designed control panels for users and supervisors to ensure safer outcomes. Autonomous aerial drones become more effective as remotely piloted aircraft and NASA's Mars Rovers can make autonomous movements, but there is a whole control room of operators managing the larger picture of what is happening.

Humans in the Group; Computers in the Loop. While people are instinctively social, they benefit from well-designed computers. Some designers favor developing computers as collaborators, teammates, and partners, when adding control panels and status displays would make them comprehensible appliances. Machine and deep learning strategies will be more widely used if they are integrated in visual user interfaces, as they are in counterterrorism centers, financial trading rooms, and transportation or utility control centers.

Explainable AI (XAI) and Comprehensible AI (CAI). Many researchers from AI and HCI have turned to the problem of providing explanations of AI decisions, as required by the European General Data Protection Regulation (GDPR) stipulating a "right to explanation."¹³ Explanations of why mortgage applications or parole requests are rejected can include local or global descriptions, but a useful complementary approach is to prevent confusion and surprise by making comprehensible user interfaces that enable rapid interactive exploration of decision spaces.

More at: https://cacm.acm.org/magazines/2021/8/254306-responsible-ai/fulltext

By Sally Ward-Foxton

Embedded-vision technologies are giving machines the power of sight, but today’s systems still fall short of understanding all the nuances of an image. An approach used for natural-language processing (NLP) could address that.

Attention-based neural networks, particularly transformer networks, have revolutionized NLP, giving machines a better understanding of language than ever before. This technique, which is designed to mimic cognitive processes by giving an artificial neural network an idea of history or context, has produced much more sophisticated AI agents than older approaches that also employ memory, such as long short-term memory and recurrent neural networks. NLP now has a deeper level of understanding of the questions or prompts it is fed and can create long pieces of text in response that are often indistinguishable from what a human might write.

Attention can certainly be applied to image processing, though its use in computer vision has been limited so far. In an exclusive interview with EE Times, AI expert Steve Teig, CEO of Perceive, argued that attention will come to be extremely important to vision applications.

Attention-based networks

The attention mechanism looks at an input sequence, such as a sentence, and decides after each piece of data in the sequence (syllable or word) which other parts of the sequence are relevant. This is similar to how you are reading this article: Your brain is holding certain words in your memory even as it focuses on each new word you’re reading, because the words you’ve already read, combined with the word you’re reading right now, lend valuable context that help you understand the text.

Teig’s example is:

The car skidded on the street because it was slippery.

As you finish reading the sentence, you understand that “slippery” likely refers to the street and not the car, because you’ve held the words “street” and “car” in memory, and your experience tells you that the relevance connection between “slippery” and “street” is much stronger than the relevance connection between “slippery” and “car.” A neural network can try to mimic this ability using the attention mechanism.

The mechanism “takes all the words in the recent past and compares them in some fashion as a way of seeing which words might possibly relate to which other words,” said Teig. “Then the network knows to at least focus on that, because it’s more likely for ‘slippery’ to be [relevant to] either the street or the car and not [any of the other words].”

Attention is therefore a way to focus on reducing the sequence of the presented data to a subset that might possibly be of interest (perhaps the current and previous sentences only) and then assigning possibilities to how relevant each word is likely to be.

“[Attention] ended up being a way of making use of time, in a somewhat principled way, without the overhead of looking at everything that ever happened,” Teig said. “This caused people, even until very recently, to think that attention is a trick with which one can manage time. Certainly, it has had a tremendously positive impact on speech processing, language processing, and other temporal things.

Much more recently, just in the last handful of months, people have started to realize that maybe we can use attention to do other focusing of information.”

Vision transformer

Neural networks designed for vision have made very limited use of attention techniques so far. Until now, attention has been applied alongside convolutional neural networks (CNNs) or used to replace certain components of a CNN. But a recent paper by Google scientists¹ argues that the concept of attention is more widely applicable to vision. The authors show that a pure transformer network, a type of network widely used in NLP that relies on the attention mechanism, can perform well on image-classification tasks when applied directly to a sequence of image patches. The transformer network built by the researchers, Vision Transformer (ViT), achieved superior results to CNNs but required fewer compute resources to train.

While it may be easy to imagine how attention applies to text or spoken dialogue, applying the same concept to a still image (rather than a temporal sequence such as a video) is less obvious. In fact, attention can be used in the spatial, rather than the temporal, context here. Syllables or words would be analogous to patches of the images.

Teig’s example is a photo of a dog. The patch of the image that shows the dog’s ear might identify itself as an ear, even as a particular type of ear that is found on a furry animal, or a quadruped. Similarly, the tail patch knows it is also found on furry animals and quadrupeds. A tree patch in the background of the image knows that it has branches and leaves. The attention mechanism asks the ear patch and the tree patch what they have in common. The answer is: not a lot. The ear patch and the tail patch, however, do have a lot in common; they can confer about those commonalities, and then maybe the neural network can find a larger concept than “ear” or “tail.” Maybe the network can understand some of the context provided by the image to work out that ear plus tail might equal dog.

“The fact that the ear and the tail of the dog are not independent allows us to have a terser description of what’s going on in the picture: ‘There is a dog in the picture,’ as opposed to, ‘There’s a brown pixel next to a grey pixel, next to …’ which is a terrible description of what’s going on in the picture,” said Teig. “This is what becomes possible as the system describes the pieces of the image in these semantic terms, so to speak. It can then aggregate those into more useful concepts for downstream reasoning.”

The eventual aim, Teig said, would be for the neural network to understand that the picture is a dog chasing a Frisbee.

“Good luck doing that with 16 million colors of pixels,” he said. “This is an attempt to process that down to, ‘There’s a dog; there’s a Frisbee; the dog is running.’ Now I have a fighting chance at understanding that maybe the dog is playing Frisbee.”

A step closer

Google’s work on attention in vision systems is a step in the right direction, Teig said, “but I think there’s a lot of room to advance here, both from a theory and software point of view and from a hardware point of view, when one doesn’t have to bludgeon the data with gigantic matrices, which I very much doubt your brain is doing. There’s so much that can be filtered out in context without having to compare it to everything else.”

While the Google research team’s solution used compute resources more sparingly than CNNs do, the way attention is typically implemented in NLP makes networks like transformers extremely resource-intensive. Transformers often build gigantic N × N matrices of syllables (for text) or pixels (for images) that require substantial compute power and memory to process.

“The data center guys out there think, ‘Excellent — we have a data center, so everything looks like a nail to us,’” said Teig, and that’s how we’ve ended up with NLP models like OpenAI’s GPT-3, with its 175 billion parameters. “It’s kind of ridiculous that you’re looking at everything when, a priori, you can say that almost nothing in the prior sentence is going to matter. Can’t you do any kind of filtering in advance? Do you really have to do this crudely just because you have a gigantic matrix multiplier? Does that make any sense? Probably not.”

Recent attempts by the scientific community to reduce the computational overhead for attention have reduced the number of operations required from N2 to N√N. But those attempts perpetuate “the near-
universal belief — one I do not share — that deep learning is all about matrices and matrix multiplication,” Teig said, pointing out that the most advanced neural network research is being done by those with access to massive matrix multiplication accelerators.

Teig’s perspective as CEO of Perceive, an edge-AI accelerator chip company, is that there are more efficient ways of conceptualizing neural network computation. Perceive is already using some of these concepts, and Teig thinks similar insights will apply to the attention mechanism and transformer networks.

“I think the spirit of what attention is talking about is very important,” he said. “I think the machinery itself is going to evolve very quickly over the next couple of years … in software, in theory, and in hardware to represent it.”

Is there an eventual point where today’s huge transformer networks will fit onto an accelerator in an edge device? In Teig’s view, networks like GPT-3’s 175 billion parameters — roughly 1 trillion bits of information (assuming 8-bit parameters, for the sake of argument) — are part of the problem.

“It’s like we’re playing 20 questions, only I’m going to ask you a trillion questions in order to understand what you’ve just said,” he said. “Maybe it can’t be done in 20,000 or 2 million, but a trillion — get out of here! The flaw isn’t that we have a small 20-mW chip; the flaw there is that [having] 175 billion parameters means you did something really wrong.”

Reducing attention-based networks’ parameter count, and representing them efficiently, could bring attention-based embedded vision to edge devices, according to Teig. And such developments are “not far away.”

By Egil Juliussen

Autonomous vehicle (AV) startups have raised billions of dollars to develop and test their self-driving technology. They will need far more to continue developing, testing and eventually deploying various AV use-cases. There is more venture capital available, but at some point most of the AV startups will need to become public companies. There are now two routes they can take — traditional IPO or via a SPAC.

Special purpose acquisition companies (SPACs), have gained a lot of attention lately both in the electric vehicle industry and among AV startups. The SPAC is becoming a key mechanism for AV startups to survive the long development and deployment phase before becoming commercial enterprises.

This column will explore the following questions:

• What is a SPAC?
• Why are SPACs becoming important to AV startups?
• Which AV startups are on the SPAC train?
• Which AV startups are using traditional IPOs?

The table below summarizes what SPACs are, their growing popularity and their advantages compared to traditional IPOs. I used some information from an excellent Harvard Business Review article called: SPACs: What You Need to Know (hbr.org)

What is a SPAC?

SPACs are publicly traded corporations formed with the sole purpose of completing a merger with a private company to enable it to go public. There are two phases to be successful: making a SPAC IPO and finding a merger partner.

The SPAC IPO is organized by a sponsor that needs considerable expertise to be successful. The sponsor makes a business plan and provides funding ( a few million dollars) for the operation of the SPAC. Next the sponsor must raise capital from investor — usually in the range of $200 to $700 million. Then the SPAC goes public and the shares are traded on stock exchanges.

In the merger phase, the SPAC management finds a private company as a suitable merger partner. The SPAC has two years to reach a merger agreement. If no merger agreement is made, the sponsor can either seek an extension or return all invested funds to the investors. If the SPAC returns the investments, the sponsor loses its capital used for operating the SPAC.

After the merger agreement, the SPAC starts a road show to confirm the valuation and raise additional capital which is called PIPE or Private Investment in Public Equity. More work is needed to finalize agreements, get approvals of investors and complete the merger. Investors can withdraw their capital during this phase and often a percentage of investors do so.

SPACs have been around in various forms for decades but were poorly regulated initially. SPAC regulations are much better now and are likely to see more improvements. The Harvard Business Review article believes today’s SPAC ecosystem is fundamentally distinct from the one that existed as recently as 2019, characterized by different risks, stakeholders, structures and performance.

In the last two years SPACs have taken off in the U.S. In 2019, 59 SPACs were started, with $13 billion invested for an average of $220 million per SPAC. In 2020, 247 were created, with $80 billion invested or an average of nearly $324 million per SPAC. In the first quarter of 2021, 295 SPACs were started, with $96 billion invested for an average of $325 million per SPAC. In 2020, SPACs accounted for over 50% of new publicly listed U.S. companies.

However, the SPAC market has slowed in 2Q 2021 and there are indications that the SEC (Securities and Exchange Commission) will be scrutinizing SPAC activities more closely in the future. This should cool down excessive actions and will be good for quality SPAC activities.

SPAC benefits

Compared with traditional IPOs, SPACs often offer a private company higher valuation, less dilution, and less time to become a public company. SPAC also offer more certainty and transparency and have lower fees and fewer regulatory restrictions.

There are additional benefits to AV startups in using a SPAC to go public. SPACs has the potential to solve the funding problem for technology startups with long-term product development requirements. AVs are in this category along with quantum computing and others. Essentially, SPACs can take over when the VC industry has done its part, or the startup want to retain a higher share or control of its ownership when large additional funding is required.

In a SPAC merger filling, the startup’s long-term business plan is included and gives the investors perspectives on future operational performance. In traditional IPOs such long-term business plans are not allowed. Instead the bankers that are handling the IPOs, provide such long-term business information via their analysts.

The startup that is part of the SPAC merger should know its business better than the bankers’ analyst and hence should provide a better projection of its future business prospects. But this is a two-edged sword and making an unrealistic business plan in the SPAC filing will be a big mistake.

SPAC risks

There are risks in using a SPAC — especially in an overheated and volatile marketplace. Some SPACs will merge with startups that will fail or do poorly. Some of the SPAC sponsors may be inexperienced and lead to poor financial results. Some SPAC investors may lose money. An April post from SEC has good information on SPAC risks: SEC.gov | SPACs, IPOs and Liability Risk under the Securities Laws

There are two IPO events in the life of a SPAC. The first IPO happens when the sponsors create the SPAC with investors buying the SPAC shares. The second event is the merger with a private company—usually a startup. Between the two events lots of changes are possible that can impact the risk range of the SPAC investment.

AV SPAC & IPOs

The next table is a summary of AV startups that have gone public using SPACs or intend to. TuSimple is the only AV company that has done a traditional IPO and it is included in the table. More information on the companies are included below the table. Note that much of the data in the table are projections when SPAC merger was announced and could change by closing time.

Aeva Technologies

Aeva is a lidar startup founded 2017 with a focus on Frequency Modulated Continuous Wave (FMCW) lidar. The lidar specs are quite strong. Aeva received VC funding of $48 million before SPAC started. Aeva’s partners include Denso, TuSimple, VW, ZF and one unnamed OEM.

Aeva announced a SPAC merger with the InterPrivate Acquisition Corp. on November 2, 2020. The merger was completed on March 15, 2021. Aeva raised an impressive $560 million from the SPAC and PIPE funding.

Aurora Innovation

Aurora Innovation is a leading developer of AV software platforms. Aurora is active in multiple AV use-cases including robotaxis, autonomous trucks and goods AVs. It acquired Uber’s AV group which is covered here: Breaking down Aurora-Uber ATG Deal | EE Times. Since its founding in 2017, Aurora has raised $1.22B in venture capital.

Aurora has agreed to go public via a merger with Reinvent Technology Partners Y, a SPAC led by LinkedIn co-founder Reid Hoffman and Zynga founder Mark Pincus. The implied valuation of the merger is $13 billion, and Aurora is expected to raise $2 billion from the deal.

This is likely to be an important SPAC event for the AV industry. It is important that this SPAC is successful for the future of AV SPACs.

AEye

AEye is a leading lidar company with added camera data. AEye was founded in 2013 and has raised $89 million in VC funding. On February 17, 2021, AEye will merge with CF Finance Acquisition Corp. III in a deal that values the company at $2 billion. AEye is projected to raise $455 million when the deal closes. The SPAC investors include GM Ventures, Subaru-SBI, Intel Capital, Hella Ventures and others.

Embark Trucks

Embark is developing an AV software platform for autonomous trucks. Embark is planning to use a SaaS business model where it charges a fee per mile driven for AV software and hardware. It was founded in 2016 and has $117 million in VC funding. On June 23, 2021, Embark and a public SPAC, Northern Genesis 2, announced a merger that value the combined company at $5.16 billion and will raise $614 million when the merger closes.

Innoviz Technologies

Innoviz is another lidar startup that was founded in 2016 in Israel. It received over $250 million from VCs before the SPAC started. Investors include Aptiv, Magna, Samsung and SoftBank. Partners include BMW and three Tier 1s—Aptiv, Harman and Magna.

Innoviz completed a SPAC merger with the Collective Growth Corporation and began trade on NASDAQ on April 6, 2021. The SPAC merger provided Innoviz with $380 million in additional capital.

Luminar

Luminar is a leading lidar company that was founded in 2012. It received over $250 million in VC funding before its SPAC—including investments from Volvo Car and Daimler Trucks.

On August 24, 2020, Luminar announced it was merging with a SPAC, Gores Metropoulos. At that time a post-deal market valuation of $3.4 billion was projected. The merger was completed on December 3, 2020 and Luminar raised $420 million. Luminar’s lidar revenue was $14 million in 2020—up 11% from 2019.

Ouster

Ouster is another lidar startup company that has gone public via a SPAC. Ouster was founded in 2015 and received $90 million in VC funding. Ouster’s revenue was nearly $19 million in 2020 and projects sales exceeding $33 million in 2021. Ouster is providing lidars to a variety of segments including multiple AV use-cases, ADAS, robotics, mining, warehouse and port-shipping automation.

On December 22, 2020 Ouster agreed to go public via a SPAC merger with Colonnade Acquisition Corp. The valuation at that time was $1.9 billion. The merger was completed on March 12, 2021 and Ouster raised around $300 million in additional capital.

Plus

Plus is a developing an AV software and hardware platform for autonomous trucking. Plus was founded in 2016 and has received $520 million in VC funding. Plus is using an aftermarket business model where it provides the hardware and software for the autonomous driving system that can be used on most existing trucks. It is testing autonomous trucks in China and U.S. and plan future testing in Europe.

Plus is planning to merge with Hennessy Capital V in SPAC agreement that was announced on May 10, 2021. The SPAC deal give Plus a valuation of $3.3 billion and will raise about $500 million when the merger closes.

In June 20212, Plus received an order from Amazon to purchase at least 1,000 Plus autonomous truck retrofit units. As part of the order, Amazon received warrants for 20% of Plus’s outstanding stock.

Quanergy

Quanergy is a lidar supplier that was founded in 2012 and has received $135 million in VC investments. Quanergy and CITIC Capital Acquisition Corp. entered into a SPAC agreement on June 22, 2021. The SPAC deal give Quanergy a valuation of $1.1 billion and is expected to raise $278 million when the merger closes in second half of 2021. Quanergy has over 350 customers and 40 partnerships globally.

Velodyne Lidar

Velodyne was the pioneer in lidar technology and was founded in 1983. It’s lidar technology was developed in 2005 for the DARPA challenge. Velodyne received around $225 million in VC funding. Ford, Baidu and Hyundai Mobis were leading investors.

Velodyne and Graf Industrial Corporation announced a SPAC agreement to merge on July 2, 2020. This was the first SPAC in the AV market. The merger closed on September 30, 2020. The valuation at IPO was $4.0 billion and $150 million was raised.

Velodyne revenue was $95 million in 2020, $101 million in 2019, $143 million in 2018 and $182 million in 2017. It looks like Velodyne’s revenue has been falling from declining lidar prices and increasing competition.

Since the Velodyne IPO, there has been a falling out between the Velodyne founder and current management and lawsuits have been filed. It is difficult to determine if the lawsuits are due to the SPAC activities or disagreement on how to reverse Velodyne’s declining revenue.

Traditional IPOs

TuSimple is a leading AV software platform for autonomous trucks. It was founded in 2015 and has received over $640 million in VC funding. TuSimple had nearly 900 employees worldwide in April 2021.

TuSimple was the first AV company to go public via a traditional IPO. TuSimple started trading on NASDAQ on April 15, 2021. Its valuation was $8.5 billion at that time. TuSimple raised $1.35 billion at its IPO. TuSimple has autonomous truck operations in U.S. and China.

Summary

I found ten AV related companies that have competed a SPAC IPO or have started a SPAC IPO — seven lidar startups and three AV software platform startups. The three AV software platform companies will become public in 2H 2021. Five lidar startups have completed their SPAC IPOs while two more have started the process.

There are five lidar companies that have SPAC IPO history. It is interesting that all of them are currently trading lower than the stock price at their IPO date.

Even if the SPAC IPOs from five lidar companies have not performed well from stock price perspectives, they have provided the lidar companies with a lot of capital that will give them a better shot for long-term success — some more than others.

This indicates that more future SPACs will happen for other AV startups. The investors are likely to be more cautious—at least the investors with short-term horizons. The success of the AV market segments are all long-term opportunities and remain very large future prospects. Such large new opportunities rarely come along in any industry.

Accerion’s Triton sensing technology for mobile robots and automated guided vehicles delivers sub-millimeter localization accuracy within operating environments, but the system does not require navigation infrastructure such as QR markers, tape, magnets laser reflectors or extra Lidar features.

Primary Target Markets – Logistics / Supply Chain, Manufacturing, Healthcare

Technology / Product / Service(s) – Accerion manufactures localization systems for Autonomous Mobile Robots (AMRs). In September 2020, Accerion introduced Triton, the world’s first 100% infrastructure-free, sub-millimeter accuracy localization technology for mobile robots operating in dynamic environments. Triton requires no embedded guiding wires, no laser reflectors, no QR codes, and no extra features for LiDAR.

To begin using the Triton system, the warehouse area is first mapped with the Triton sensor by recording images of the floor. After recording, artificial intelligence is applied to convert the recorded images to unique “signatures.”

In operation, Triton maps the “signatures,” then provides coordinates to the navigation system so mobile robots can localize themselves in the environment.

Triton provides absolute localization updates as input for drift corrections of the vehicle’s navigation system. Using Triton, the floor becomes a  map, and there is no additional infrastructure required for precise localization.

Triton’s sub-millimeter-level accuracy enables mobile robots to perform high-performance operations in dynamic environments at a speed of 1.5 to 2 meters per second.

Value Proposition – Now more than ever, E-commerce companies need to automate – and scale quickly – to keep up with demand and stay ahead of the competition. Accerion’s positioning system ticks off many of the boxes for logistics operators – especially Goods to Person – who want to accelerate automation. AMRs play a key role in many Goods to Person order fulfillment operations as they provide a modular, scalable platform on which to build the process.

Accerion’s positioning systems enable AMRs to follow a virtual grid with sub-millimeter accuracy at high speed and operate in highly dynamic environments… without any infrastructure requirements. This means Accerion’s Triton positioning system is the right technology at the right time to further accelerate industrial automation, giving Goods to Person operators a competitive advantage.

To summarize:

·        Triton needs no infrastructure. The floor is your map, which eliminates hefty installation costs and maintenance of localization infrastructure such as induction wires, magnetic tape, laser reflectors or QR markers.

·        Triton’s sub-millimeter repeat accuracy results in faster AMR driving, faster docking and more effective fleet operation.

·        Triton offers flexibility and scalability. After mapping the warehouse floor, operators can easily scale E-commerce business by adding more AMRs, more SKUs, and more storage and mobile racking. Using the existing floor surface, the virtual grid offers flexibility and scalability so logistics operations can grow quickly.

·        Triton gives AMRs high-speed driving capability at 1.5 to 2 meters per second.

·        Last but not least, Triton is economical. Triton’s dynamic and accurate vehicle operation means operators can do more with fewer AMRs, significantly reducing costs.

This white paper takes an in-depth look at machine tending operations within today’s factories, and how cobots help machine shops free up valuable personnel, increase capacity while improving quality, and breathe new life into idle equipment.

More details at: https://www.cobottrends.com/cobot-special-ur?spMailingID=50035&puid=1243846&E=1243846&utm_source=newsletter&utm_medium=email&utm_campaign=50035

By Gina Roos

At one time, Synaptics Inc. was best known for its interface products, including fingerprint sensors, touchpads, and display drivers for PCs and mobile phones. Today, propelled by several acquisitions over the past several years, the company is making a big push into consumer IoT as well as computer-vision and artificial-intelligence solutions at the edge. Synaptics sees opportunities in computer vision across all markets and recently launched edge-AI processors that target real-time computer-vision and multimedia applications.

The company’s recent AI roadmap spans from enhancing the image quality of high-resolution cameras using the high-end VS680 multi-TOPS processor to serving battery-powered devices at a lower resolution with the ultra-low–power Katana Edge AI system-on-chip (SoC).

Last year, Synaptics introduced the Smart Edge AI platform, consisting of the VideoSmart VS600 family of edge-computing video SoCs with a secure AI framework. The SoCs combine a CPU, NPU, and GPU and are designed specifically for smart displays, smart cameras, video sound cards, set-top boxes, voice-enabled devices, and computer-vision IoT products.

The platform uses the company’s Synaptics Neural Network Acceleration and Processing (SyNAP) technology, a full-stack solution for on-device deep-learning models for advanced features. With inferencing done on the device, it addresses privacy, security, and latency issues.

The VS600 SoCs include an integrated MIPI-CSI camera serial interface with an advanced image-signal–processing engine for edge-based computer-vision inference. The also use the company’s far-field voice and customizable wake-word technology for edge-based voice processing and the SyKURE security framework.

At the other end of the spectrum is an ultra-low–power platform for battery-operated devices. Built on a multicore processor architecture optimized for ultra-low power and low latency for voice, audio, and vision applications, the Katana Edge AI platform features proprietary neural network and domain-specific processing cores, on-chip memory, and use of multiple architectural techniques for power savings. Katana Edge AI can be combined with the company’s wireless connectivity offerings for system-level modules and solutions.

“There is a ton of applications where plugging in is just not viable, so there is an interest in battery power, whether it is in the field or industrial, and particularly at home,” said Patrick Worfolk, senior vice president and chief technology officer at Synaptics. “With this particular Katana platform, we’re targeting very low power.”

Typical applications for the Katana SoC for battery-powered devices include people or object recognition and counting; visual, voice, or sound detection; asset or inventory tracking; and environmental sensing.

The Katana platform also requires software optimization techniques coupled with the silicon, which is where the company’s recently announced partnership with Eta Compute comes into play. The Katana SoC will be co-optimized with Eta Compute’s Tensai Flow software, and the companies will work together to offer application-specific kits that will include pre-trained machine-learning models and reference designs.

Users will also be able to train the models with their own datasets using frameworks such as TensorFlow, Caffe, and ONNX.

Synaptics eased its path into consumer IoT via two acquisitions in 2017: Conexant Systems LLC and Marvell Technology Group’s Multimedia Business Unit. Conexant gave the company access to advanced voice- and audio-processing solutions for the smart home, including far-field voice technology for trigger-word detection and keyword spotting, while Marvell’s Multimedia Business Unit delivered extensive IP for advanced processing technology for video and audio applications, particularly digital personal assistants, as well as the smart home.

With the Conexant acquisition, Synaptics gained a portfolio of audio products, providing the right architecture to do keyword spotting at the edge — and that is done through neural networks, said Worfolk. The multimedia team carved out from Marvell Technology was developing video processors, and those devices are used in streaming media as well as in smart displays, he added.

“As these smart display products integrate cameras, you run into all the same challenges around performance and privacy, and there’s more and more drive to do those algorithms in the edge device,” said Worfolk. “The natural structures for those types of algorithms today with the best performance are AI algorithms.”

In 2020, Synaptics bolstered its IoT position with the acquisition of Broadcom’s wireless IoT business, adding Wi-Fi, Bluetooth, and GNSS/GPS technologies for applications including home automation, smart displays and speakers, media streamers, IP cameras, and automotive. By pairing its edge SoCs with the wireless technology, it can open up opportunities beyond the consumer IoT market.

Synaptics also acquired DisplayLink Corp., adding its universal docking solutions and video-compression technology to its portfolio. The company will combine the video-compression technology with its existing and new video interface products and new wireless solutions.

Built on edge processing

Processing at the edge is not new to Synaptics. All processing of the raw data in its embedded sensing chips, including fingerprint products and touch controllers, happens on-chip because of concerns around power, latency, and security.

Even before Synaptics manufactured its first interface products, the company was founded in 1986 to research neural networks but pivoted into other technologies before coming full circle to develop edge-AI processors for computer-vision and multimedia applications.

“We were founded to do neural network chips over 30 years ago; back then, all the chips were analog AI chips, and it was challenging to scale well,” said Worfolk. “In fact, the company went off in a slightly different direction after the initial founding and started doing pattern recognition, which is a classic AI problem. We’ve been doing AI for a long time, but we have recently migrated to deep learning, and these deep neural networks have really taken over by storm.

“With the breakthroughs in AI in the last decade, more and more of these traditional algorithms have mapped over to AI algorithms, enabling performance advantages when you do the processing at the edge,” he added.

Worfolk said the nature of the company’s products and the vertical markets it serves are driving the need for AI-based algorithms.

“We’ve entered the AI space vertically through our existing markets, and then with those products, we’re expanding into neighboring markets,” he said. “This is quite different from many of the startups you’ve seen in this space who have some sort of novel concept about some kind of AI processing and are developing a chip that they want to go broadly across multiple markets.”

Next steps

In the early days, Synaptics developed its own algorithms for its chips. The keyword-spotting and trigger-word algorithms for voice digital assistants, as examples, are the company’s core algorithms. But Synaptics wanted to open up its silicon to allow third parties to run their own deep neural networks and other algorithms on its chips, so it needed a tool suite. That is not so easy to do.

The company entered into a partnership with Eta Compute to develop the software tools to train a deep neural network and compile it “so it can run on our silicon, and we could move a little faster and open up our chips to third parties,” said Worfolk.

There are other challenges in a market where innovation is happening at such a fast pace, which often can lead to performance tradeoffs.

“The field as a whole is very immature, and in that sense, it is moving very quickly; there are new announcements about new types of neural networks every single week, and a lot of the work has been done through academic or big research groups that are trying to push the boundaries of performance,” said Worfolk. “But there is a big gap between academic research and what can actually run on a small device. Although we are seeing algorithms that are able to perform on new levels that we’ve never seen before on the vision or the audio side, they take more and more compute.”

This often translates into tradeoffs between efficiency and flexibility. What typically happens is the first piece of silicon for a particular target market isn’t very flexible, and as the neural networks that “run on that silicon mature and become more stable to produce the desired functionality, we can look at a second-generation chip,” which is optimized for that performance, said Worfolk.

“It’s what makes it so exciting for us, because there’s always something new and interesting, but it is a challenge from a business perspective,” he said.

The company’s strategy is to build the silicon and then create an example application or demo that makes it easier to discuss with customers. “It also generates ideas around what you can do with the chip and makes sure that we understand what it takes to bring this chip to product,” said Worfolk.

At the Embedded Vision Summit, Worfolk demonstrated the Katana chip in a battery-powered people counter used to track usage in office conference rooms.

“This system is not just the Synaptics Katana chip; it also includes cameras, motion sensors, and wireless communications, which are also part of our portfolio,” he said. “If you want to run on, for example, four AA batteries for two years, Katana is a platform that, under appropriate use cases and conditions, could operate on that kind of time frame.”

Worfolk also showcased the more powerful VS680 multimedia SoC in an AI-based scaling application. The demo showed how the chip can be used for super-resolution enhancement, upscaling from full HD to 4K using a neural-network–based upscaler, which shows crisper and sharper images than are possible with a traditional hardware-scaling algorithm.

“There is a lot of specsmanship in silicon, but at the end of the day, you want to know if your deep neural network runs efficiently or effectively on the device or not,” he said. “So the goal of the presentation [was] to discuss sample applications that can run on Synaptics devices.”

So how do you select the example application? “As we do our MRD [market requirements document] and PRD [product requirements document] for the chip, there are particular applications that we target, those that we view as flagship applications for the piece of silicon, and that drives the demo,” said Worfolk. “We want something that is sufficiently challenging to show off the competitive advantages, but we also want a demo that has broad interest and is representative of what customers might want to do.”

An example is the people counter for the conference room, which shows off the chip’s ability to run an object-detection network at relatively low power, Worfolk said.

The Katana SoC is based on the company’s custom NPU architecture and proprietary DSPs. It also includes an off-the-shelf CPU and DSP that makes it easy for its customers to run their algorithms on the chip, he said. This is coupled with Eta Compute’s tool chain, which makes it easy for customers to port their networks.

“We gain a great deal of efficiency for the particular networks we have in mind by architecting our own MPU solutions, so we have all the right features in our silicon, and then we add the NPU that is tailored for what we expect some of those verticals will need and make it broad and flexible enough to go into adjacent markets,” said Worfolk.

“I suspect that many companies have fairly similar architectures, so it really is a matter of sizing the compute engines, the memory, and then the interfaces to the sensors,” he said. “When you know what neural network you want to run and the resources it requires, you can make sure that you don’t run into any bottlenecks. So there is a bunch of co-engineering between the very bottom, [which is] the hardware design; the top, [which is] the neural-network–model architecture; and then the tools that map that neural-network–model architecture all the way down to running on the hardware. By considering all those together, that is where you can have a real competitive advantage.”

A lot of companies don’t have their own AI teams, so they are learning about AI even as they look to integrate it into their products, Worfolk said. To support such companies, Synaptics has partners who can either train or optimize the models or support the tools.

He views partnerships as a way to fill the company’s gaps, particularly in the IoT space, where there is a broad range of applications and customers who do not have expertise in AI.