Prompt Engineering FoundationsModern assistant features: modes, memory, research and media · Lesson 9 of 16

Prompting with files, photos, screenshots and voice

Article · 12 min · 8 min lecture

Video lecture

Prompting with files, photos, screenshots and voice

12 chapters · about 8 min · captions · full transcript

Watch now

Chapters

Assistants can see, read and listen

Most major assistants now accept more than typed text. You can upload PDFs, spreadsheets and slides; take a photo with your phone; share a screenshot; or simply talk. The same prompting principles apply: say what the material is, what you want done with it, and what good output looks like.

Files: PDFs, documents and spreadsheets

When you upload a file, the assistant reads its text and, in many products, looks at page images too, which helps with tables and charts. Good habits:

  • Say what the file is and why you uploaded it. "This is our 2026 staff handbook. I need to answer employee questions about annual leave."
  • Ask for page or section references so you can check answers.
  • Narrow the job: "Using only section 4" is more reliable than "Summarise everything".
  • For spreadsheets, ask the assistant to calculate using its analysis tools rather than estimate, and check a few numbers yourself.
Attached is our supplier contract (PDF, 18 pages). I need to know:
1. The notice period to end the contract.
2. Any penalties for late payment.
3. Whether prices can change during the contract.
For each, quote the exact clause and give the page number. If the
contract doesn't say, write "Not stated". Keep the answer under 200 words.

Photos and screenshots

A phone photo or screenshot can save a lot of typing:

  • Receipts and invoices: "Extract the date, supplier, total and currency into a table. Write UNREADABLE for anything you can't read clearly."
  • Error messages: "This is a screenshot of an error in our booking software. What does it mean and what should I try first?"
  • Whiteboards and handwritten notes: "Turn this whiteboard photo into a tidy action list with owners."
  • Products and places: "What kind of plant is this and how often should I water it indoors in Dubai's climate?"
  • Design feedback: "Here are two versions of our flyer. Which is easier to read from two metres away, and why?"

Tips: crop to what matters, take photos flat and in good light, and label multiple images ("Image 1 is the old menu, image 2 the new one"). Before sharing, blur or crop out anything private: faces of people who have not agreed, names, addresses, card numbers.

Voice mode

Voice mode lets you talk to the assistant and hear its reply. It is excellent for:

  • practising interviews, presentations or a new language;
  • brainstorming while walking or driving (hands-free and safely);
  • getting quick explanations when typing is awkward.

Prompting by voice works best when you set the scene first: "I'm going to practise a sales call. You play a busy restaurant owner who is sceptical about social media ads. Keep your replies short. After five minutes, give me feedback." Some assistants can also look through your phone camera during a voice conversation and talk about what they see; check the feature and its privacy settings before using it with other people around.

Worked example: a clinic manager in Karachi

A clinic manager photographs a handwritten weekly rota, uploads the staff leave policy PDF, and asks:

Image: this week's handwritten staff rota.
File: our leave policy.
1. Type the rota into a table (Day | Morning | Evening). Mark anything
   unreadable as "?".
2. Check it against the policy: flag anyone scheduled on approved leave
   days or working more than 5 shifts.
Quote the policy rule for each issue.

The assistant produces the table and flags two issues. The manager confirms the two "?" entries with the staff directly and corrects one misread name. Total time: ten minutes instead of forty.

What assistants still get wrong

  • Blurry or tiny text may be misread or even invented. Ask for UNREADABLE instead of guesses.
  • Long documents: details in the middle can be missed. Ask for page references and check.
  • Charts: assistants describe trends well but can misread exact values. Use the underlying data when you have it.
  • Counting many small objects in a photo is unreliable.

Hands-on: three multimodal tasks

Try one task of each type this week:

1. FILE: upload a policy, contract or report and ask three questions,
   requiring a quote and page number for each answer.
2. PHOTO: photograph a receipt or handwritten note and ask for a table,
   with UNREADABLE for anything unclear.
3. VOICE: use voice mode to practise a 3-minute conversation you have
   coming up, then ask for feedback on clarity and confidence.

How to know you are doing it well

For files and photos: every important answer can be traced to a page or part of the image, and you checked it. For voice: you finish the session with specific feedback you can act on, not just a pleasant chat.

Key takeaways

  • Say what the file or image is, why you are sharing it and what output you need.
  • Ask for page references or UNREADABLE markers so answers can be checked instead of guessed.
  • Crop, label and de-identify images before sharing; blurry text and exact chart values are common failure points.
  • Voice mode is ideal for practice, brainstorming and hands-free help when you set the scene first.

Check your understanding

Quick questions to lock in the lesson. They don’t count towards your certificate.

  1. You upload a 40-page contract. Which request is most reliable?
  2. A receipt photo is faded. What should your prompt ask the assistant to do with unclear values?
  3. What should you do before sharing a screenshot of a customer's account screen?

Put it into practice

This week, complete one file task (with page-referenced answers), one photo task (with UNREADABLE markers) and one voice practice session, and note what you had to check or correct.

Enrol for free to save your progress

Reading is always free. Enrol to keep your place, take the final assessment and earn a verifiable certificate.