2021 Annual Letter — Sri Anumakonda
My goal with this letter is to show how I’ve spent the past 365 days understanding how I can level myself up, and executing on those ideas. I’ve had >75 mentor calls and met some of the most inspiring people on the planet who’ve not only shifted my trajectory, but showed me that there’s more than one lens you can view the world in (s/o to Shreyas for showing me the best ways to maximize growth and for showing me what it means to create legitimate projects).
I’ve failed more this year than the other 14 years combined from building DataGAN to networking to even my sleep. And yet those failures were the ones that taught me more about myself than any experience I ever could have.
Through failure comes learning. Through failure comes understanding. Through failure comes appreciation. Through failure comes excitement towards my future.
This year has not only been the most rewarding one, but also the most enjoyable year I’ve lived so far.
This letter will mainly be centred around these 3 things:
- an overall summary + highlights of what I’ve done this year
- my failures
- my game plan to make 2022 the best year yet
Looking through the eyes of a camera; seeing the world as a self-driving car
01/10/2021 — I created a presentation explaining my progress with autonomous vehicles (AVs) and computer vision, something quite foreign but also something I really wanted to do.
So I built something: using semantic segmentation to predict lane lines. I remember crying at 12am on the day of the presentation after spending the past 6 hours restarting the project. I spent 2 weeks building it all up but had to restart because I couldn’t figure out why my model wasn’t learning. And now the new model still wasn’t learning properly. Fast forward and I eventually got decent results after overfitting. But the presentation itself? It was mediocre.
Failure hits fast. Failure hits hard. The more important thing though was that I restarted the project from scratch and was able to completely rebuild it.
The art of hustling through and pushing yourself to break that wall that’s stopping you from moving forward is truly beautiful. I learned so much about U-Net, implementing segmentation networks, and just creating neat code stored across multiple .py files.
Perseverance gives you the keys to the doors that you thought were always locked.
Becoming one of the youngest self-driving engineers IN THE WORLD
In February now, filled with so much momentum after not only rebuilding the lane detection project but also meeting David, one of the smartest people that I’ve ever met in self-driving. I didn’t know what was next until he told me that Udacity runs the biggest certified self-driving program in the world.
The first thing I did coming off that call was making my Udacity account and signing up for the nanodegree. This ended up not being possible because of unforeseen circumstances. But I knew this was the thing that I wanted to do. I ended up taking a massive leap of faith and asking David, right on the call if there’s any way I could do the nanodegree via a scholarship.
After 15+ emails back and forth and talking with Patrick [shoutout to you personally Patrick! super grateful for the effort you’ve put into this], I was able to do the nanodegree, start to finish, in 12 weeks. I spent >5 hours every single day not only understanding the math behind algorithms but also debugging code and writing reflection pieces.
Where did that lead me?
One of the proudest achievements that I’ve received so far. It was surreal to not only learn so much within a matter of a few weeks but also to the people that helped me and allowed me to pursue something that I’ve wanted to do for so long.
What else happened? I had the chance to meet Sebastian Thrun himself! It was a dream come true. I’ve looked up to Sebastian for the past 12 months with the pioneering work he’s done with DARPA, but also his willingness to play long-term games with the goal of making an impact.
I’ve never felt more ecstatic coming out of a meeting ever, in my life. Just being able to spend 20 minutes with someone who has not only changed the world but has provided opportunities for millions to learn via Stanford University and Udacity was amazing.
I also was featured on Udacity for the work I’ve been doing:
And then came summer…
I’ve gotten so much momentum. Not only finishing the nanodegree and meeting some remarkable people like Alexander has gotten me super hyped for the summer. I remember thinking about what I wanted to do and I eventually broke this down into 2 main things: fully understand high school math, and build a research project.
I haven’t read many papers in the first half of 2021. Why? Because I didn’t understand that many of them. So I decided to reimplement a paper. What would I end up reimplementing? Lane Detection. How? By leveraging LSTM networks. I remember reading this paper on Robust Lane Detection from Continuous Driving Scenes and I knew it was something I wanted to do, especially how well it intertwines with the lane detection project I built earlier this year.
I spent 2 months implementing the paper. Doing this in private repo’s, completely doing this in stealth. Yes, I had some really neat-looking code that was not only useable but also showed a successful implementation of their paper. The only problem? No one knew. Nobody even knew what I was building throughout all of summer and I made the assumption that people did.
Building in public is the most valuable skill that I’ve learned in 2021. Showcasing progress creates an online presence that topples the domino effect for serendipity.
The other mistake that I made was not getting an internship. I’ve spent the majority of 2021 in research and understanding the gap between academia and industry. To build in the industry, you should be in the industry to understand + find problems in it. Understanding what sorts of algorithms are used in developing self-driving cars would’ve allowed me to build more meaningful projects that can genuinely be used.
I’m currently looking for summer ’22 internships in Computer Vision, robotics, or in self-driving! Shoot me an email if you’re intersested.
I’ve gotten so many slumps throughout the past year because of online school causing me to feel exhausted. The root cause? Transitioning between school and other parts of my day has been difficult. I’ve concluded that the majority of times where I’ve felt tired come from times when I’m trying to switch from one task to another.
Working out has allowed me to ease that transition drastically. I sometimes like going on a quick 15min walk to just do reflections and post-mortem examinations on the work I’m building and the session that I’m coming out of.
I also moved houses over the summer. It doesn’t sound like much of a big deal but finally being able to have my own, proper room, with books, a bed, a whiteboard, is amazing. Here’s a pic of a sunrise drive that I took!
Understanding next-gen self-driving tech w/ Waabi!
Tawsif told me about Prof. Urtasun and her self-driving startup that came out of stealth. I was literally amazed. I just stared at their YouTube videos and printed > 70 pages worth of papers to read. GeoSim and TrafficSim are examples of some of the most advanced simulation research I’ve ever seen with self-driving cars.
I sent cold emails to the authors of these papers and even had the chance to meet them! I not only had the chance to learn from the smartest people I’ve ever met in my life (h/t to Alex + Andrei!) but also was able to implement their advice and see month-by-month growth:
Waabi would undoubtedly be one of the places that would be a dream come true for me to intern at. I believe that an AI-first approach will not only allow for robust modelling + deployment but being able to learn from the smartest people in self-driving would be a dream come true.
Creating next-gen ways to train AVs w/ DataGAN
I started building DataGAN after my chat with Andrei in September. I personally found GANs to be not only something that interested me but also was a whole realm of Computer Vision that I had yet to explore.
And what’s the best way to understand GANs? BUILD THEM! I started DataGAN: Leveraging Synthetic Data for Self-Driving Vehicles as a way to create synthetic data that can be used to train robust neural agents.
Whenever I generally build projects, I love building in the intersection of awesome tech and in a field that I don’t know much of. For the project back in summer with LSTMs (ConvLSTMs) in lane detection, I didn’t know much about spatiotemporal scene understanding. With DataGAN, I knew nothing about GANs. So I spent the first 3 weeks understanding them and the next 5 weeks building this out and getting validation. The result?
I even had the chance to meet Seung, who’s built some of the COOLEST work I’ve ever seen with GANs in my life. He’s awesome, we had an amazing chat and being able to jam out about GANs in self-driving was awesome.
I eventually decided to drop the project. I’ve spent >250 hours planning this project out, understanding GANs + the math behind them, and yet I decided to drop everything. Why? GANs are notoriously hard to train. I’ve sent >150 emails regarding compute issues to labs and research institutes in Toronto but I didn’t even get a single response. I was stagnant, for 2 whole weeks.
It doesn’t make any sense to sit around and expect serendipity to show up before you. Serendipity is not a genie stored in a lamp; you cannot expect good things to happen if you don’t put in the work. I liked the idea of going on sprints like DataGAN where I’d level up my understanding in a foreign field. So I decided to build in a space that I’ve always wanted to understand: end2end learning.
Hopping on the Wayve of end2end learning
2 reasons why I wanted to get into end2end learning. 1) I thought it was fascinating how neural networks can map from image → steering, and 2) I had a meeting with Haibo in a few days.
Meeting Haibo and learning about Wayve [h/t Satvik!] was amazing. I woke up at 5:30AM to have our meeting (which was at 6:00) and looking back at it, getting 45 minutes of her time to jam out on end2end learning was amazing. Definitely one of the best meetings that I’ve personally had.
I came out of that meeting knowing for a fact that I wanted to learn more about e2e systems. I’ve printed, annotated, and handwritten notes for >250 pages worth of papers within the space of 4 weeks and started prototyping + building end2end models.
Interpreting neural networks is one of the hardest problems that there is right now. It’s crucial to be able to understand these networks and the weights activations to better understand the “rationale” for a certain prediction made. I was specifically looking into seeing how we can use CNNs [and ConvLSTMs] to better understand + extract spatiotemporal context given images, along with using autoencoders to predict the next few frames in a driving scene given the predicted speed + steering (n=1 is what’s used in the diagram). Proposal can be viewed here.
I also got >1.35k in AWS credits to build this project out which ultimately kickstarted the “paper instinct” and now I wanted to turn this idea into a large-scale project that I could test, validate, and write a paper on.
I spent 2 weeks playing around with the comma dataset + udacity to see if I can really train a network for steering, speed, and next-frame prediction. It didn’t make sense for me to go that far for the sake of the paper. I realize now that even small-scale testing provides nearly the same benefits as building large-scale projects. I wanted to build a large-scale project so that I can show people that I wrote a paper and made a cool video + animations. But that didn’t fundamentally add value to the actual idea itself.
Just as in the case of DataGAN, by not creating accurate + quantifiable metrics, I wasn’t able to make proper progress and eventually decided it was best to take a break from this.
Dropping 2 projects in 2 months
There are 2 reasons why I dropped these projects:
- Assumptions that I’ve made
- The value prop from building this
The first problem that I had (with both projects) was the fact that I made assumptions. I spent the whole time assuming that a professor or grad student would magically respond to my email, let me borrow a PC with a GPU for research purposes (or even work at a lab!), and everything would be perfect. I thought that I’d be able to write a paper on DataGAN and the end2end idea that I’ve been playing around with. I looked at both of my projects from an ideal standpoint, not something that was even realistic.
This has been my biggest regret of 2021. Not realizing that making hypothetical assumptions for projects that are completely outside of my scope won’t bring any sort of progress. Instead, explore! Do sprints and deep dives of different fields in Computer Vision.
I stopped DataGAN because I realized that it didn’t make sense for me to continue pursuing it. I wasted so much time sitting back and waiting for the emails to pop in. What’s the value in that when I can start learning calculus, learning data structures, and even doing programming challenges?
end2end learning is a bit different. This is a goal that I want to pursue long-term. I want to actually build end2end systems. I think end2end might be the way we truly achieve level 5 driving. I wanted to understand end2end systems and then build a paper.
end2end learning is a bit different. This is a goal that I want to pursue long-term. I want to actually build end2end systems. The 4-week “sprint” that I thought of doing was to understand end2end learning systems and then build a paper (here’s the proposal). I dropped the proposal because again, it didn’t make sense + I didn’t have the compute to train dense neural networks.
I’m starting to build in this closed-loop system of having idea sprints + building small-scale projects. This is probably the best way for me to not only get technical understanding but also be able to show legitimate progress.
Failing hard
The biggest thing that I’ve done this year is failing. Not just in the projects that I’ve built, but also in the way I chose to grow as a person. I’ve lost momentum several times, but that’s allowed me to not only understand myself better but also allow me to understand the goals and impact that I want to make on this world.
Lane Detection — one of the first failures that I’ve gotten in 2021. I’ve failed by not allocating enough time to build this project enough, optimized for creating “professional” github repos instead of focusing on getting the model working, and also failed in not choosing to reach out to mentors + people who might’ve been willing to help me out. The main takeaway here was that code does not always work; understanding + learning to debug problems is a crucial aspect in developing intuition behind the code you write. I’ve allowed myself ever since to explore + create neat code that people can easily reimplement and also just focus on understanding the goal. I didn’t fully understand U-Net before I dived into it, and as a result, I wasn’t able to accurately debug it in time.
The biggest thing I’ve learned over the summer though is how important it is to set clear intentions, KPIs, and how to really break down your time in chunks that allow you to explicitly see week-by-week growth. I’ve failed in that aspect completely. I woke up every single day without clear intentions of what I wanted to achieve and just spent 10 hours typing up as much code as I can and expected it to work. I’ve become so much more productive when I have clear-cut intentions set out for the day.
I’ve also failed a lot in DataGAN. I set unrealistic expectations without a defined path that would get me there. I made claims such as making a paper and doing large-scale testing, but I never sat down and brainstormed how I’d achieve what I wanted. I expected the puzzle pieces to fit themselves together without realizing that I’m the person that brings the pieces together. I need to complete the puzzle, it doesn’t happen itself.
I failed in over-optimizing for computational power because I wanted to build large-scale projects. I wasted so much time waiting for an email when I could’ve been making so much more progress instead by creating a small-scale implementation and then moving on. I failed in assuming that the research paper is the end goal and serendipity stems from that outcome.
I’ve failed in taking opportunities for granted. I’ve failed in being passive. I’ve failed in understanding the world. I want to do better. I will do better. 2021 has shown me what my weakness really are. I want to strengthen my weaknesses and be able to learn from them.
How I can make 2022 the best year
I’ve learned a lot. I’ve failed a lot. I want to make next year the best year I could ever possibly live in. There are 3 main buckets that I want to optimize for 2022:
- Become a self-driving “god”
- Get an internship
- personal health
The first thing that I’m going to be focusing on is really developing my T. I’ve done that a lot this year. I learned so much about AI, Computer Vision, and even built my depth of expertise with the projects that I’ve been building. But I want to take it further. I want to start reading textbooks and understand so much more about self-driving + the fundamental principles of the components of self-driving [specifically looking into linear algebra, the mathematics of ML, and planning algorithms for AVs].
I want to research, build, and showcase projects in these fields:
- motion planning
- reinforcement learning + policy development
- creating variational autoencoders for image-to-image translation
- end2end learning + conditional imitation learning
There’s so much to learn, despite the fact that I’ve scoped down from AI → self-driving → computer vision. I’ve been super passionate about computer vision and yet there’s still so much to learn and understand. How do things like optical motion flow impact the perception of a self-driving car? Can we simply use Q-Learning with CNNs to train robust agents?
The main focus of all of this will be to understand fundamental concepts and the principles of mathematics and programming. This will be the most compounding skill that I’ll have right now. Understanding math is a superpower. Being able to understand the fundamental understanding of an algorithm, concept, and paper will not only allow me to comprehend lower-level details i.e. optimization algorithms + custom loss functions, but also will allow me to question the math and understand why it works.
Right now, I’m reviewing calculus + differentiation. I’m also thinking about reading “Mathematics for Machine Learning” to become more fluent with linear algebra + applying the math in various ML algorithms. I’m going to be focusing on playing long-term games i.e. learning math. This is the best way for me to really make an impact in the summer and will bring my technical understanding to a whole new level.
The other focus of 2022 will be to get an internship where I can build next-gen self-driving tech. Getting the chance to learn from some of the smartest people in the industry will not only allow me to better understand self-driving as a whole, but it’ll really push my technical understanding. I specifically want to work in the end2end learning space, looking into the interpretability of neural networks and even build end2end systems myself!
Some of the companies on my radar right now are, without question, Waabi + Wayve. Both of them have been pioneering some really cool AI-first research. I’d love to play around with cameras and understand how we can use VAEs and spatiotemporal networks to allow for robust performance when driving.
Another possible route would be to get into is a research lab here in Toronto ex. Vector! Some of the smartest people in self-driving are usually found in research labs (we have a lot of smart people in Toronto!) and being able to network + work with them on interesting research projects would be great.
The third thing that I really want to double down on for 2021 will be personal health. Something that I’ve neglected for so long. I want to focus on not only sleep + working out, but also nutrition.
It’s not just about physical changes, but mental too. Even going on 15min walks is a huge refresher for transitions throughout the day. And the best thing is? You feel happy. You feel grateful for where you are and you feel motivated to strive to do more.
The main KPIs I’m currently looking at are upkeeping proper sleep, developing close relationships with the people who I talk to, and maintaining physical health. I’ll be focusing on strength + cardio training along with sports!
This is what I want 2022 to be all about. It’s not just about the number of hours that you put in working, but it’s about how well you perform in those hours.
Words cannot describe how grateful I really am for the growth I’ve had over the past 12 months. Want to personally shout out Sebastian, Shreyas, Rahul, Andrei, Alex, Haibo, David, Oliver, Navid, Michael, Harrison, Kim, Dan, and Nadeem for the mentorship + guidance you’ve provided me with. The amount of time you’ve spent helping me level up myself is something that not only will I be eternally grateful for but also the mentorship and the endless experiences you’ve provided me with.
Also shoutout to Mikael, Zayn, Aaahan, Dev, Arnav, Anush, Kabeer, Pavan, Sriya, and Ahmed for not only allowing me to accelerate on my trajectory but also for the insightful conversations that we’ve had all this time. The successes [and failures!] that I’ve gotten this year wouldn’t have come without the calls that we’ve had. I’ve learned a ton about the projects you’ve all been working on from fUS to computational biology and it’s just been simply awesome nerding out with you guys!
Concluding thoughts
Being able to have a room to myself, a computer to type this letter on, and fingers to type has allowed me to do things that not many 15yo really think about. I’ve had countless opportunities that people my age would only be able to dream about.
Even typing this letter out has allowed me to reflect on my year and better understand myself a lot better than ever before. I’ve learned, failed, and grown in so many aspects. I’ve met some of the smartest people that I’ve ever met in my life.
I want to crush 2022. I want to optimize to make it the best year ever. I’ve gained so much momentum + trajectory over the past 12 months and I want to keep accelerating. Why be linear when you can be exponential?
Hey! 👋 If you’d like to follow along with the progress that I’ll be making in 2022, here’s my newsletter! Twitter too. You can also reach out to me on my LinkedIn + email, free say to drop in and say hi! Oh, and personal website too.
