Why is AI alignment urgent?

• Lucky = arms race, unlucky = war with CCP
  • Self explanatory, international laws have never been much good at regulation even if voluntary (see climate change) because they have no jurisdictive power.

• So GPT 2 to 4 took us from preschooler to smart highschooler in 4 years, we’re not observing diminishing capabilities per compute, so we should expect a similar jump in capabilities in another 4 years.
  • Data constraints and algorithmic efficiency gives this large error bars
  • Hobbled on certain tasks and excel at others due to pre-training data distribution but general capabilities are there
  • How? Deep learning + MSA just works lol
• We’re (USA) out here regulating models based on COST OF COMPUTE
  • Apparently no one involved knew about Moore’s law
• Unhobbling:
  • Giving them tools, long term memory, access to computer, embodiments, reasoning mechanisms, personalization/specialization
• We’re also racing through orders of magnitude of compute (just look at Nvidia) to reap economic gains from AI applications. So there is reason to place high probability density of AGI in this decade
• Once we hit AGI (being able to do AI R&D) intelligence explosion is likely (considering OpenAI is considering to use this for alignment, it can just as easily be used for capabilities)
  • We only need to automate AI research for it to impact every field…