AI can process vast quantities of data, and without much noteworthy human intervention, transform it into an AI-generated output. The discussion on how to treat any intellectual property rights arising in both the materials used to train the AI (input) and the results created by the AI (output) is still in its early days.

To keep these issues comprehensible, we focus here on legal questions concerning copyright laws, but the same concepts are likely to be applicable to other sorts of protected IP.

Materials used to train the AI (input)

Depending on the law of the relevant jurisdiction, the materials used to train the AI could be copyright protected, and it is likely that reproductions of these materials are made during the training process. Unless certain exceptions to copyrights could be invoked, these kinds of reproductions may constitute an infringement to the copyrights of the author of these materials. These exceptions will vary from jurisdiction to jurisdiction. For example, in the United States, there is a concept of a “fair use” exception, whereas in the EU, the exceptions for transient or incidental copying and text and data mining may be relevant.

Therefore, it is difficult to identify which materials could be used to train an AI system without infringing any IP rights, including copyrights. The recent US Supreme Court ruling in the Warhol case on fair use, which focused more on the commercial purpose of new works than on the artistic expression, is likely to complicate the assessment of US-related copyright risks of AI training materials. However, the ruling’s tangible repercussions are not clear yet and will likely be decided in the lower courts.

AI-generated output as a copyright-protected work

Broadly, current copyright law grants rights to the author of a protected work. The focus is on protecting the author’s intellectual and personal relationship with their work and to ensure that authors maintain control over the exploitation of their works.

However, when it comes to outputs from Generative AI, the question arises as to whether these outputs can have an author, as the composition of the output is not done by a human mind but by an AI system. Lawmakers in their particular jurisdiction will have a role in determining whether granting a copyright to the user complies with the purpose of copyright laws, not least because the user may not have made any free and creative choices that contribute in a meaningful way to the output.

For example, in the EU, the European Parliament stated, in a resolution published on October 20, 2020 that works created independently by an AI system are not currently eligible for copyright protection since intellectual property rights generally require an individual that is involved in the creation process. The AI Act does not deviate from this understanding. The US Copyright Office issued a statement in March 2023 that copyright protection does not extend to works generated by AI except to “the extent to which the human had creative control over the work’s expression and “actually formed” the traditional elements of authorship” as demonstrated in the Zarya of the Dawn case.

Additionally, in August 2023, the US District Court for the District of Columbia affirmed the US Copyright Office’s position established in Zarya of the Dawn by granting summary judgment in the Thaler v. Perlmutter case, where the US Copyright Office denied a copyright application for a work generated by a machine claiming human authorship is required for copyright protection.

Therefore, in broad terms, we may find lawmakers moving toward a position where modifying the output of an AI system and creating a new (derived) work allows the human author to obtain copyright; whereas, the more the output is created by the AI system itself, the less likely it is that such rights will attach. The implications of the Warhol case must also be taken into consideration

Generative AI systems both ingest and generate large amounts of data, including images, text, speech, video, code, business plans, and technical formulae. Training, testing, uploading, analyzing, consulting, or otherwise processing such input and output data requires various levels of protection.

Such levels of protection depend on the type of data, with a significant distinction between personal and non-personal data. When data qualifies as personally identifiable information (e.g., names, information on a person’s life), data protection laws may apply, either locally (e.g., CCPA in California) or regionally (e.g., GDPR in Europe).

Business data, such as financial and technical information, strategic knowhow, and trade secrets, may also be classified as confidential information under local laws or by contract, providing for both civil and criminal penalties in cases of mishandling. In this context, when using Generative AI systems, organizations must carefully consider proper categorization of data inputted into these systems and take steps to ensure data is processed lawfully, securely, and confidentially.

To this end, we turn to some of the main challenges organizations face when using personal and confidential data in Generative AI systems, as well as the measures they could adopt to mitigate the relevant legal risks.

Personal data roles and responsibilities

From an EU perspective, a starting point for a personal data protection assessment when using Generative AI is to consider the roles of the parties involved (i.e., data controller, data processor/service provider etc.). This helps define which entity bears the primary responsibility for compliance and what specific actions are to be taken.

In this respect, a Generative AI system provider—as per principle and in a simplified business model—would operate as the data controller for the first layers of training and testing data. Moreover, the provider would most likely operate as an independent data controller for all data, while offering off-the-shelf, data-embedded products. The provider may also act as a data processor on behalf of a customer organization for input and output data, especially where the provider simply licenses the AI “engine” to enterprise customers without any embedded data.

In both cases, the customer organization will likely operate as a data controller for any additional layers of training and testing, for input or output data, depending on the applicable business model. Mixed roles or even joint controllership are also possible and should be considered on a case-by-case basis, in the context of the required data protection and algorithmic impact assessments.

It must be noted that the aforementioned scenarios have not been ruled by any court or supervisory authority yet.

Confidentiality

A breach of confidentiality, imposed either by law or by contract, is a risk to the rights and freedoms of both people and organizations. As such, ensuring the ongoing confidentiality of data across the entire AI lifecycle is an essential factor.

Generative AI models can inadvertently learn and reproduce sensitive information present in the training data. This can result in the generation of outputs that contain confidential information, which, if shared or made public, may compromise confidentiality.

Businesses also need to be aware of their own confidentiality obligations. If a business’s use case requires confidential information that has been shared by customers, suppliers, or other third parties, the business will need to first consider any duties of confidentiality and other contract terms under which the information was shared and whether they are permitted to use of that data within a Generative AI system.

Measures to consider adopting

As the use of Generative AI continues to increase, organizations need to carefully assess the existing legal, financial, and reputational risks connected with personal data and confidentiality. Organizations may want to think about the following non-exhaustive list of considerations in addition to legal and regulatory requirements as they come into force:

• Should data access be limited to authorized personnel? What role should physical and logical access control mechanisms, such as authentication systems play?
• What specific policies and procedures for the use of Generative AI tools will be adopted and how will they be maintained, and compliance audited?
• Will policies and procedures be adapted, ensuring the exercise of individual rights (e.g., data deletion)?
• What training and awareness sessions for employees on the ethical, lawful, and secure use of this technology are appropriate?
• How do supply chain audits and controls impact organizations whether they are a supplier or recipient of AI Generative services?
• What technical and organizational measures (e.g., AI governance, privacy-by-design and by-default, pseudonymization, anonymization, encryption, and secure storage) should be put in place to ensure organizations and the personal or confidential data they ingest or retrieve are protected against unauthorized disclosure, alteration, or loss of availability?
• Will legal specialists and technologists be involved in the designing of controls to protect personal data and confidentiality from the early stages of any AI project and will the expertise be in-house or external?