Project Moonshot: Singapore’s AI Toolkit for Safer Large Language Models
At ATxAI, a signature conference on Day 2 of Asia Tech x Singapore (ATxSG) 2024, Singapore’s Minister for Communications and Information, Mrs Josephine Teo, launched AI Verify Project Moonshot (“Moonshot”), an easy-to-use testing toolkit designed to address security and safety challenges often associated with the use of large language models (LLMs), the cornerstone of many AI-driven solutions. It is one of the world’s first open-sourced tools to bring red-teaming, benchmarking, and baseline testing together in an easy-to-use platform – a testament to Singapore’s commitment to harnessing the power of the global open-source community in addressing AI risks.
An open beta, Project Moonshot aims to provide intuitive results of the quality and safety of a model or application in an easily understood manner, even for a non-technical user. It was developed through working with partners such as DataRobot, IBM, Singtel, and Temasek to ensure that the tool is useful and aligned with industry needs.
“IBM is pleased to be a design partner and contributor to AI Verify Moonshot. The provision of this new tool is significant as it aims to help developers and data scientists test their LLM applications against a baseline of risks, thereby accelerating the adoption of AI. We look forward to working closely with IMDA to develop appropriate open standards through our contributions.” – Anup Kumar, Distinguished Engineer, CTO Data & AI, Head Client Engineering Asia Pacific, IBM
“DataRobot is thrilled to partner with IMDA to make LLM evaluation more accessible. The integration of Project Moonshot into the DataRobot AI Platform allows AI builders to confidently and responsibly scale generative AI within their organisations.” – Jay Schuren, Chief Customer Officer, DataRobot.
Project Moonshot is also part of an important move towards global testing standards. Two of the leading AI testing organisations – AI Verify Foundation and MLCommons – have come together to build a common safety benchmark suite.
“We are proud that leading AI testing organisations AI Verify Foundation and MLCommons are coming together to support common testing benchmarks for AI safety, building on innovative efforts like Project Moonshot. To further the collaboration, AI Verify Foundation and ML Commons have signed a memorandum of intent (MOI) on 29th May, intending to positively impact AI safety by providing model and application developers with a globally accepted approach of safety testing for generative AI.” – Peter Mattson, MLCommons President and co-chair AI Safety working group.
AI Verify Foundation members gather for 1st anniversary celebration (photo credit: IMDA)
At the AI Verify Foundation 1st year anniversary celebration held yesterday, Dr Ong Chen Hui, Chair, Governing Committee, AI Verify Foundation, shared that key feedback from users were the need for AI Verify to align with emerging standards and cater for sector specific testing. In response to this feedback, she announced that AI Verify Foundation has completed the mapping of AI Verify with ISO 42001 and would be integrating AI Verify with MAS’ Veritas toolkit.
Championing Women in Tech in Southeast Asia
Minister Teo also spoke on a panel at Women and Youth in Tech, where she unveiled findings from an IMDA and Boston Consulting Group (BCG) joint report titled Closing Tech’s Gender Gap in Southeast Asia. The report surveyed over 4,000 individuals across the region including Singapore, Indonesia, Malaysia, the Philippines, Thailand, and Vietnam. According to the report, 83% of women in the tech sector are satisfied with their jobs and 70% believe that their company has initiatives to improve the recruitment, retention, and advancement of women – an increase from 49% in 2020. It also found that with 40% of women in technology, Singapore remains at the forefront of promoting gender diversity and inclusion in the industry within Southeast Asia. However, while there have been greater strides to support women in the sector across the region, more can be done to improve women’s representation in technical roles and leadership.
Other key highlights at ATxSG include:
Start of Global Consultation on Digital Inclusion
The United Nations Development Programme (UNDP), in collaboration with IMDA, convened a high-level roundtable at the ATxSummit and launched a 100-day Global Consultation and Digital Inclusion Playbook. The Playbook features IMDA’s Digital Skills for Life (DSL) framework1, which underpins the national Digital for Life (DfL) movement, among other global case studies. With Singapore as the starting point, the roundtable kickstarted the consultation process with a productive dialogue among senior government officials from various countries, as well as representatives from international organisations, the private sector and civil society
IMDA partners with Google to bring Gemini Academy’s GenAI content and resources to the community
In support of the national DfL movement, Google has rolled out the Gemini Academy, a skilling programme designed to educate learners on the fundamentals of AI. Originally designed for teachers, Singapore will be the first country where the Academy’s content is extended beyond the classrooms to the members of the public. IMDA will take reference from Gemini Academy’s curated GenAI content, in our update of the DSL resources to include GenAI content. GenAI content can be found on the DfL portal, which also features resources for learning DSL. IMDA will also work with Google’s Gemini Academy on GenAI workshops at NLB’s libraries. The public can register for these workshops later in 2024 via NLB’s Eventbrite page.
For details and the latest agenda on ATxSG, please visit this website.
