Content
- About the Company
- About the Project
- Vacancy
- Location
- Duration
- Eligibility
- Application Procedure
About the OpenNyAI
OpenNyAI works with a diverse network of legal professionals, technologists and academic institutions. The goal is to encourage the integration of AI in the justice sector by developing the necessary tools and addressing the challenges posed by the process.
About the Project
Artificial Intelligence (AI) technology are finding their way into more and more fields, including law. The purpose of RAG (Retriever Augmented Generation) systems is to improve the precision of AI responses.
AI frequently “hallucinates” or produces incorrect responses. The two primary components of a RAG system are a generator and a retriever. The generator creates a response using the data that the retriever retrieves from an already-existing database. A RAG system reduces the likelihood of hallucinations by producing more accurate answers due to its small database.
However, the absence of a consistent framework makes evaluating these technologies extremely difficult. The metrics used to evaluate Large Language Models (LLMs) are frequently model-specific, and thus might not be readily transferable to RAG systems. This emphasizes the necessity of standardized testing procedures, particularly in native AI development environments devoid of such frameworks.
In order to tackle the difficulties associated with AI assessment, our initiative seeks to establish an extensive legal benchmarking dataset in India. RAG systems will be evaluated using this dataset for recall and accuracy, two important parameters.
In order to tackle the difficulties associated with AI assessment, our initiative seeks to establish an extensive legal benchmarking dataset in India.
Accuracy evaluates the information retrieved’s accuracy, whereas recall gauges the system’s capacity to recover all pertinent data. A varied group of students from the humanities, social sciences, law, gender studies, and data sciences will work together to create this dataset. The intention is to cultivate a dataset that, by incorporating a variety of viewpoints and levels of skill, aids in the advancement of Responsible AI.
We are seeking volunteers to assist with compiling legal datasets as part of the OpenNyAI objective. The RAG system is the AI framework whose accuracy will be evaluated using this dataset. We need you to complete the job (about 15 minutes) listed below in order to express your interest.
Vacancy
200 people
Location
Online
Duration
2-3 hours every weekday from 20th July to 20th August,2024
Eligibility
- Law Students: To participate in our annotating exercises, as a significant portion of the work requires legal aptitude. We are preferably looking for 2nd – 5th year law students.
- Students from various disciplines: Including social sciences, gender studies, data sciences, engineering, and humanities. We aim to create a dataset that lends itself to Responsible AI, and we’re looking for as much diversity as possible.