Amazon Augmented AI FAQs

Many machine learning applications require humans to review low confidence predictions to ensure the results are correct. For example, extracting information from scanned mortgage application forms can require human review in some cases due to low-quality scans or poor handwriting. But building human review systems can be time consuming and expensive because it involves implementing complex processes or “workflows”, writing custom software to manage review tasks and results, and in many cases, managing large groups of reviewers.

Amazon A2I makes it easy to build and manage human reviews for machine learning applications. Amazon A2I provides built-in human review workflows for common machine learning use cases, such as content moderation and text extraction from documents, which allows predictions from Amazon Rekognition and Amazon Textract to be reviewed easily. You can also create your own workflows for ML models built on Amazon SageMaker or any other tools. Using Amazon A2I, you can allow human reviewers to step in when a model is unable to make a high confidence prediction or to audit its predictions on an ongoing basis.

Amazon A2I provides a managed experience where you can set up an entire human review workflow in a few easy steps. To get started with Amazon A2I, sign in to your AWS Console, and navigate to the Amazon SageMaker console. From there, select Human review workflows under Augmented AI. First, as a part of the human review workflow, you provide a pointer to the S3 bucket where the review results should be stored. Next, you select the appropriate task type and define conditions when a human review should be triggered. Amazon A2I provides pre-built workflows where you only need to enter a few choices and provide instructions on how your objects should be reviewed by humans. Alternatively, you can create your own custom workflow and use your own custom review templates. Once created, the workflow can be used directly in your applications using a generated unique identifier for this workflow.

With A2I, you can define what is an acceptable prediction confidence for your business problem. You can define business rules for the machine learning predictions, based on which a human review is triggered. For Amazon Rekognition image moderation tasks, you can use the confidence score that Amazon Rekognition provides for each label it outputs to trigger human review. For Amazon Textract tasks, you can trigger a human review when specific form keys are missing or when form key detection confidence is low. You can also trigger a human review if, after evaluating all form keys in the text, confidence is lower than your required threshold for any form key. For your own custom workflow, you can write the code for business conditions in AWS Lambda or directly in your client application.

With Amazon A2I, you can choose from three workforce options: (1) Amazon Mechanical Turk; (2) Third party data labeling service providers available through the AWS Marketplace; and (3) Your own employees. See the Amazon A2I developer guide for more information.

Please see the Amazon A2I pricing page for the current pricing information.

The AWS Region Table lists all the AWS regions where Amazon A2I is currently available.

Yes, Amazon Augmented AI service providers can process customer confidential data. The Standard Service Agreement between AWS customers and the third-party service provider contains clauses to protect your confidential information. Please review those terms before sharing any confidential information with the service provider. The terms are located on the listing page for the service provider on AWS Marketplace.

In light of the rapidly evolving impact of COVID-19, some service providers have implemented a remote work policy for the health and safety of their employees temporarily. During this time, security standards including SOC 2 compliance and additional security controls outlined in the below FAQ may not be applicable to the affected service providers. Impacted service providers have updated their AWS Marketplace listings to reflect this, and will not process customer data from remote work environments without explicit customer consent.

Human review service providers are required to go through SOC 2 compliance and certification on an annual basis. The SOC 2 report is a description of the service provider’s control environment based on the American Institute of Certified Public Accountants (AICPA) Trust Services Criteria - Security, Availability, Processing Integrity, Confidentiality, and Privacy.

In addition to SOC 2, service providers are required to maintain these additional security controls to secure customer data in their environments.

Technology Controls:
Service providers are required to utilize the appropriate software to block any attempts to download or copy files/data from their system and prevent unauthorized access to their systems. Service providers are also required to prohibit their workforce from storing or copying customer task-related data outside of service provider secure environments.

Network Security Controls:
Service providers must prohibit remote access to customer's task-related data. Further, peer-to-peer file sharing software is blocked on the provider's network.

Employee Controls:
Service providers are required to ensure they have Non-Disclosure Agreements (NDAs) with their employees. Service providers are required to adopt stringent policies to prevent prevent employees from copying or moving customer task related data from providers secure environment including controls for paper, USBs, mobile phones, or other media.

Physical Access Controls:
Service providers are required to maintain physical access control measures to prevent unauthorized access to their production site. These may include bio-metric authentication, employee badge identification, visual verification of employees by security guards, etc.

AWS requests that service providers furnish their SOC 2 certification reports prior to being listed in the marketplace and confirms:

Authenticity (if service provider auditor is certified by the AICPA);

Report period (SOC 2 certification validity date); and

Production site verification (the physical site where the service provider workforce will work on Amazon Augmented AI tasks).

The security standards from every service provider are reviewed annually to ensure they meet the mandatory requirements.