Software

AI system devises first optimizations to sorting code in over a decade

Taran 09 June 2023

Anyone who has taken a basic computer science class has undoubtedly spent time devising a sorting algorithm—code that will take an unordered list of items and put them in ascending or descending order. It’s an interesting challenge because there are so many ways of doing it and because people have spent a lot of time figuring out how to do this sorting as efficiently as possible.

Sorting is so basic that algorithms are built into most standard libraries for programming languages. And, in the case of the C++ library used with the LLVM compiler, the code hasn’t been touched in over a decade.

But Google’s DeepMind AI group has now developed a reinforcement learning tool that can develop extremely optimized algorithms without first being trained on human code examples. The trick was to set it up to treat programming as a game.

It’s all a game

DeepMind, among other things, is notable for having developed software that teaches itself how to play games. That approach has proven highly effective, conquering games as varied as chess, Go, and StarCraft. While the details vary depending on which game it’s tackling, the software learns by playing itself and discovers options that allow it to maximize a score.

Because it isn’t trained on games humans play, the DeepMind system can discover approaches to the games that humans haven’t thought of. Of course, since it’s always playing against itself, there are cases where it has developed blind spots that humans can exploit.

This approach is very relevant to programming. Large language models write effective code because they have seen plenty of human examples. But because of that, they’re unlikely to develop something that humans haven’t done previously. If we’re looking to optimize well-understood algorithms, like sorting functions, then basing something on existing human code is, at best, going to get you equivalent performance. But how do you get an AI to identify a truly new approach?

The people at DeepMind took the same approach as they had with chess and Go: They turned code optimization into a game. The AlphaDev system developed x86 assembly algorithms that treated the latency of the code as a score and tried to minimize that score while ensuring that the code ran to completion without errors. Through reinforcement learning, AlphaDev gradually develops the ability to write tight, highly efficient code.

Inside AlphaDev

Saying that the system optimizes for latency is very different from explaining how it operates. Like most other complex AI systems, AlphaDev consists of several distinct components. One of them is a representation function, which tracks the overall performance of the code as it’s developed. This includes the general structure of the algorithm, as well as the use of x86 registers and memory.

The system adds assembly instructions individually, chosen by a Monte Carlo tree search—again, an approach borrowed from game-playing systems. The “tree” aspect of this approach allows the system to quickly narrow in on a limited area of the large range of potential instructions, while the Monte Carlo adds a degree of randomness to the precise instruction that gets chosen from that branch. (Note that “instruction” in this context includes things like the specific registers chosen to create a valid and complete assembly.)

The system then evaluates the state of the assembly code for latency and validity and assigns it a score, comparing that to the score of the previous one. And, through reinforcement learning, it hangs on to information about how going down different branches of the tree work, given the program’s state. Over time, it “learns” how to achieve a winning game state—a completed sorting—with a maximum score, meaning a minimum latency.

The main benefit of this system is that its training doesn’t have to involve any code examples. Instead, the system generates its own code examples and then evaluates them. In the process, it hangs on to information about combinations of instructions that are effective in sorting.

Useful code

Sorting in complex programs can handle large and arbitrary collections of items. But at the level of standard libraries, it’s built from a large collection of highly specific functions that handle just one or a few situations. For example, there are separate algorithms for sorting three items, four items, and five items. And there’s another set of functions that can handle an arbitrary number of items up to a limit—meaning you can call one that sorts up to four items, but no more.

DeepMind set AlphaDev on each of these functions, but they operate very differently. For the functions that handle a specific number of items, it’s possible to write code without any branches where you execute different code based on the state of a variable. As a result, the performance of this code generally scales with a number of instructions required. AlphaDev was able to shave an instruction off of sort-3, sort-5, and sort-8, and even more off of sort-6 and sort-7. There was only one (sort-4) where it didn’t find a way to improve the human code. Repeated runs of the code on actual systems showed that fewer instructions did lead to better performance.

Sorting a variable number of entries does involve branching in the code, and different processors have different amounts of hardware dedicated to handling these branches. So for these, the code was evaluated based on its performance on 100 different machines. Here again, AlphaDev found ways to squeeze out additional performance, and we’ll take a look at how it did this in one situation: a function that sorts up to four items.

In the existing implementation in the C++ library, the code does a series of tests to see how many items it needs to sort and calls the dedicated sorting function for that number of items. The revised code does something much weirder. It tests if there are two items and calls out to a separate function to sort them if needed. If it’s greater than two items, the code calls out to sort the first three items. If there are three items, it returns the results of that sort.

If there are four items to sort, however, it runs specialized code that is extremely efficient at inserting a fourth item into the appropriate place within a set of three sorted items. This sounds like a weird approach, but it consistently outperformed the existing code.

In production

Since AlphaDev did produce more efficient code, the team wanted to get these incorporated back into the LLVM standard C++ library. The problem here is that the code was in assembly rather than C++. So, they had to work backward and figure out the C++ code that would produce the same assembly. Once that was done, the code was incorporated into the LLVM toolchain—the first time some of the code had been modified in over a decade.

As a result, the researchers estimate that AlphaDev’s code is now executed trillions of times a day.

by John Timmer for arstechnica.com

92 Replies to “AI system devises first optimizations to sorting code in over a decade”

  6. Hey! I could have sworn I’ve been to this site before but after reading through some of the post I realized it’s new to me. Nonetheless, I’m definitely happy I found it and I’ll be bookmarking and checking back often!

  10. I like the helpful information you provide in your articles.
    I’ll bookmark your weblog and check again here regularly.
    I am quite sure I will learn many new stuff right here! Good luck for the next!

  11. Great post. I used to be checking continuously this blog and I’m inspired!
    Very helpful info particularly the final section 🙂 I handle such info a lot.
    I was seeking this certain information for a very long time.
    Thank you and good luck.

  12. Hello there! I know this is kinda off topic but I’d figured I’d ask.
    Would you be interested in trading links or maybe guest writing a blog
    article or vice-versa? My site covers a lot of the same subjects as yours and I feel we could greatly benefit from each other.
    If you’re interested feel free to shoot me an email.
    I look forward to hearing from you! Terrific blog by the way!

  14. Greetings from Los angeles! I’m bored to tears at work so I decided to check out your website on my iphone during lunch break. I enjoy the info you present here and can’t wait to take a look when I get home. I’m amazed at how quick your blog loaded on my mobile .. I’m not even using WIFI, just 3G .. Anyhow, amazing site!

  16. We are a gaggle of volunteers and starting a new scheme in our community. Your web site provided us with helpful information to work on. You have performed an impressive activity and our whole community might be grateful to you.

  17. Hi there would you mind stating which blog platform you’re working with? I’m planning to start my own blog in the near future but I’m having a difficult time selecting between BlogEngine/Wordpress/B2evolution and Drupal. The reason I ask is because your design and style seems different then most blogs and I’m looking for something completely unique. P.S Apologies for getting off-topic but I had to ask!

  21. Hey there! Would you mind if I share your blog with my twitter group? There’s a lot of people that I think would really enjoy your content. Please let me know. Thanks

  23. Hey there! Do you know if they make any plugins to help with SEO? I’m trying to get my blog to rank for some targeted keywords but I’m not seeing very good gains. If you know of any please share. Thanks!

  24. Hi there! This article couldn’t be written any better! Reading through this post reminds me of my previous roommate! He always kept talking about this. I will forward this information to him. Pretty sure he’ll have a good read. Many thanks for sharing!

  32. Great article! This is the type of information that are meant to be shared around the web. Disgrace on the seek engines for now not positioning this submit upper! Come on over and talk over with my site . Thank you =)

  34. Hey there! I know this is kinda off topic however I’d figured I’d ask. Would you be interested in exchanging links or maybe guest writing a blog article or vice-versa? My site goes over a lot of the same subjects as yours and I believe we could greatly benefit from each other. If you happen to be interested feel free to send me an e-mail. I look forward to hearing from you! Awesome blog by the way!

  39. Hello there I am so thrilled I found your web site, I really found you by error, while I was searching on Digg for something else, Regardless I am here now and would just like to say thanks a lot for a marvelous post and a all round interesting blog (I also love the theme/design), I dont have time to look over it all at the minute but I have saved it and also added in your RSS feeds, so when I have time I will be back to read a lot more, Please do keep up the superb b.

  43. Someone necessarily help to make significantly articles I would state. This is the first time I frequented your web page and so far? I amazed with the research you made to create this actual post incredible. Wonderful process!

  44. It’s really a cool and helpful piece of information. I’m glad that you simply shared this helpful info with us. Please stay us informed like this. Thank you for sharing.

  45. Pretty great post. I simply stumbled upon your blog and wanted to mention that I have really enjoyed browsing your blog posts. In any case I’ll be subscribing in your feed and I am hoping you write again soon!

  47. It’s a shame you don’t have a donate button! I’d without a doubt donate to this superb blog! I suppose for now i’ll settle for book-marking and adding your RSS feed to my Google account. I look forward to brand new updates and will talk about this blog with my Facebook group. Chat soon!

  48. Can I simply say what a relief to find an individual who actually knows what they’re talking about on the net. You definitely know how to bring an issue to light and make it important. More and more people should read this and understand this side of the story. It’s surprising you aren’t more popular because you certainly have the gift.

  49. I am extremely impressed with your writing skills and also with the layout on your blog. Is this a paid theme or did you customize it yourself? Either way keep up the nice quality writing, it’s rare to see a nice blog like this one nowadays.

  51. I like the valuable information you supply in your articles. I will bookmark your weblog and check again here frequently. I am rather certain I will be informed a lot of new stuff right here! Good luck for the following!

  53. An impressive share! I have just forwarded this onto a friend who was doing a little research on this. And he in fact bought me lunch simply because I discovered it for him… lol. So let me reword this…. Thank YOU for the meal!! But yeah, thanx for spending the time to discuss this issue here on your website.

  56. Hey there! This is my 1st comment here so I just wanted to give a quick shout out and tell you I genuinely enjoy reading through your blog posts. Can you suggest any other blogs/websites/forums that deal with the same subjects? Thanks for your time!

  58. I am extremely inspired together with your writing talents and alsowell as with the layout on your blog. Is this a paid subject or did you customize it yourself? Either way stay up the nice quality writing, it’s rare to see a nice blog like this one nowadays..

  70. Thanks for a marvelous posting! I seriously enjoyed reading it, you’re a great author. I will be sure to bookmark your blog and definitely will come back later in life. I want to encourage one to continue your great job, have a nice morning!

  77. Wonderful goods from you, man. I’ve understand your stuff previous to and you’re just too wonderful. I really like what you’ve acquired here, really like what you’re stating and the way in which you say it. You make it entertaining and you still take care of to keep it smart. I can not wait to read far more from you. This is actually a great website.

  80. I’m really enjoying the design and layout of your blog. It’s a very easy on the eyes which makes it much more enjoyable for me to come here and visit more often. Did you hire out a designer to create your theme? Great work!

  82. Excellent post. I was checking continuously this blog and I am impressed! Very useful information particularly the last part 🙂 I care for such info a lot. I was seeking this particular info for a long time. Thank you and good luck.

  89. My programmer is trying to persuade me to move to .net from PHP. I have always disliked the idea because of the expenses. But he’s tryiong none the less. I’ve been using Movable-type on a variety of websites for about a year and am nervous about switching to another platform. I have heard great things about blogengine.net. Is there a way I can transfer all my wordpress content into it? Any kind of help would be really appreciated!

  90. I am really loving the theme/design of your website. Do you ever run into any web browser compatibility problems? A handful of my blog audience have complained about my website not operating correctly in Explorer but looks great in Safari. Do you have any tips to help fix this issue?

Comments are closed.