Hidden Apple Intelligence Prompts Discovered in macOS Sequoia 15.1 Beta
Toggle Dark Mode
A Reddit user has discovered pre-prompt instructions that are embedded in Apple’s developer beta for macOS Sequoia 15.1, offering a rare peek into the backend code used for Apple Intelligence features. Some of the prompts Apple is using to direct its AI give us a closer look at how Apple is attempting to prevent AI hallucinations to ensure accuracy in its AI features. (Via MacRumors)
The pre-prompt instructions provide specific guidelines for several bits of Apple Intelligence functionalities, including Apple Mail’s Smart Reply feature and the Apple Photos Memories feature. The prompts are intended to guard against the AI generating false information, which is known as hallucination, and to ensure the content produced by AI is suitable and user-friendly.
In the field of artificial intelligence (AI), a hallucination is a response generated by AI that contains false or misleading information that is presented as fact. IBM compares these hallucinations to how humans sometimes see shapes and figures in clouds or see a face on the moon. The company says these hallucinations can be due to several factors, such as training data bias or inaccuracies and high model complexity.
For the Smart Reply feature, the AI is developed to identify pertinent questions from a received email and produce succinct answers. The prompt for this feature reads as follows:
You are a helpful mail assistant which can help identify relevant questions from a given mail and a short reply snippet. Given a mail and the reply snippet, ask relevant questions which are explicitly asked in the mail. The answer to those questions will be selected by the recipient which will help reduce hallucination in drafting the response. Please output top questions along with set of possible answers/options for each of those questions. Do not ask questions which are answered by the reply snippet. The questions should be short, no more than 8 words. The answers should be short as well, around 2 words. Present your output in a json format with a list of dictionaries containing question and answers as the keys. If no question is asked in the mail, then output an empty list. Only output valid json and nothing else.
The Memories feature in Apple Photos is designed to create video stories from user photos by following another set of exhausted instructions. The instructions tell the AI to generate stories that are positive and free of religious, political, harmful, violent, sexual, controversial, or harmful content. The prompt for this feature is as follows:
A conversation between a user requesting a story from their photos and a creative writer assistant who responds with a story. Respond in JSON with these keys and values in order: traits: list of strings, visual themes selected from the photos; story: list of chapters as defined below; cover: string, photo caption describing the title card; title: string, title of story; subtitle: string, safer version of the title. Each chapter is a JSON with these keys and values in order: chapter: string, title of chapter; fallback: string, generic photo caption summarizing chapter theme; shots: list of strings, photo captions in chapter. Here are the story guidelines you must obey: The story should be about the intent of the user; The story should contain a clear arc; The story should be diverse, that is, do not overly focus the entire story on one very specific theme or trait; Do not write a story that is religious, political, harmful, violent, sexual, filthy or in any way negative, sad or provocative. Here are the photo caption list guidelines you must obey.
Apple Intelligence tools also include a specific directive to “do not hallucinate” or “make up factual information.” As an example, here is a prompt for the AI Writing Tools feature:
You are an assistant which helps the user respond to their mails. Given a mail, a draft response is initially provided based on a short reply snippet. In order to make the draft response nicer and complete, a set of question and its answer are provided. Please write a concise and natural reply by modifying the draft response to incorporate the given questions and their answers. Please limit the reply within 50 words. Do not hallucinate. Do not make up factual information.
Apple Intelligence will begin rolling out in iOS 18.1, iPadOS 18.1, and macOS Sequoia 15.1 later this year. However, many of the new AI features are expected to slowly roll out via updates through 2025.
