Naming what it looks like
When a photo is clear enough, Skinby names up to four conditions it looks like. I tested this on 371 photos of faces where dermatologists had listed what they saw. Skinby named something on 81% of them. When it did, a condition from the dermatologists' list was among the names 91% of the time (somewhere between 87% and 94% allowing for chance).
On the other photos, Skinby does not name anything and says so.
Some conditions look alike in a photo and are told apart by things a photo does not show. When Skinby cannot narrow the answer to four names, it can name one of four look-alike groups instead of nothing: acne or folliculitis; eczema, dermatitis (including seborrheic dermatitis) or psoriasis; hives or an insect bite; and ringworm or tinea versicolor. For the first two it then asks one or two questions that narrow the group to one name. The numbers above include these answers.
I also tested it on 2,686 ordinary portraits of people with no skin condition. It named a condition on 0.5% of them (13 of 2,686).
One check was changed after I tested whole-face photos of people with acne: the guard that stops ordinary faces from being named was hiding acne on many of them, because a full face with a few dozen small spots still looks mostly clear. For acne only, the guard is now relaxed when the model's own acne reading is clearly above its level on ordinary faces. I chose that level using ordinary portraits alone, so the rate above already includes the change. I have not measured acne on darker skin, and the other conditions have no whole-face photos to check against.
Acne-like spots, between scans. Skinby can say there are "more" or "fewer" acne-like spots than at your last scan or than your usual. This comes from how strongly the model recognises acne across your whole face. It is not a count of spots and not a grade, and the number is never shown. To avoid false changes I measured how far this reading moves between two ordinary photos of the same face: on 204 open-access photos of acne it moved by less than 1.5 (on the model's own log-odds scale) in 95% of repeat photos, and the change threshold is set above that, so smaller moves are not mentioned. On those photos the reading ranks them like my own rough spot counts (rank correlation 0.65); those counts are mine, not an expert's. It cannot tell moderate from very severe acne, because the model spreads very severe acne over similar conditions; it has not been measured separately for darker skin; and a real change smaller than the threshold is not mentioned.
Split by skin tone (6 groups, estimated from the photos), the share of ordinary portraits that got a condition name ranged from 0.0% to 0.7%. With this few cases the true rate in any one group could be as high as 2.4%.
Phone selfies look different from portraits, so I repeated the test on 286 of those portraits after making them look more like selfies: skin smoothing, blur, warm or cool light, dim light with noise, strong light from one side and heavy compression (8 changes in all). Skinby named a condition on at most 0.0% of the changed photos, and the colour words stayed the same for at least 0% of the readings. These are changed portraits, not real selfies; the beta measures real ones.
| Skin tone group | Photos | Named something | List matched | Range |
|---|---|---|---|---|
| Lightest skin tones (Monk 1 and 2) | 153 | 80% | 92% | 86% to 95% |
| Medium skin (Monk 3) | 108 | 84% | 88% | 80% to 93% |
| Medium to deeper skin (Monk 4 and 5) | 79 | 81% | 92% | 83% to 97% |
| Deepest skin tones (Monk 6 and above) (small sample) | 31 | 81% | 92% | 75% to 98% |
I have fewer examples for medium and deeper skin, so those numbers are less certain. Medium skin (Monk 3) did less well than the lightest group.
Serious conditions such as shingles, cellulitis and drug rashes are never named from a photo.
Eczema, the contact dermatitis types and the general "dermatitis" labels are answered together as "eczema or dermatitis", because dermatologists use those labels interchangeably. That makes the answer broader, so the numbers above are for the broader answer. Telling the types apart takes a question about what triggers it.
Limits
This compares Skinby with a panel of dermatologists looking at photos. That is agreement with their lists, not proof of what is on a person's face.
The photos come from people who chose to share a skin concern, so they are not a random sample of everyone.
I adjusted parts of Skinby while looking at these same photos, even though each photo was kept out of its own training. The numbers are likely a little optimistic, and there is no fresh set of untouched photos yet.
Measuring redness and brown pigment
To test the measurement, I added known amounts of redness and brown pigment to 240 real face photos across 6 skin-tone groups and checked whether Skinby's reading followed the amount. It did in every group: the match was between 0.88 and 0.98 on a scale where 1.00 means the reading matched the added amount exactly. Changing the light changed the reading by at most 1% of the real effect, and a smooth shadow across the face by at most 7%.
The test covered the cheeks, forehead, nose, chin and the skin under the eyes. Some areas can be read less often than others, because of hair, shadow or an uneven edge. Depending on skin tone, I could read the cheeks on 75% to 90% of the photos, the nose on 83% to 93%, the chin on 63% to 78%, under the eyes on 57% to 75% and the forehead on 53% to 78%. When an area cannot be read, Skinby says nothing about it.
Readings compare each spot with the skin around it, and then with what is usual for that area on skin of a similar tone, so a word is only said when a reading stands out. They are not a score.
When two scans of your face differ, Skinby says a reading is higher or lower only if it moved more than it moves between photos of the same skin taken in ordinary different conditions (light, blur, compression). I measured that spread on 238 faces, each photographed 8 times with such differences; a move bigger than it happens by chance in at most 5% of pairs. Because of that, small changes are not mentioned. A known added change of the largest size I tried was noticed 95% of the time for forehead redness, 87% for cheek redness and 38% for cheek brown pigment, so changes in pigment are rarely large enough to be mentioned. Darker skin tones read noisier, so each tone group has its own limit. These are changed photos of the same people, not two real sessions, so real day-to-day changes in skin make the true limit larger.
Under the eyes there is one more reading: how much darker the skin is than the skin around it. I tested it the same way, adding known darkening to 240 photos across 6 skin-tone groups. The match was between 0.99 and 1.00, a change of light moved it by at most 3% of the effect, and a smooth shadow across the face by at most 7% for nine in ten photos.
Limits
Darker under the eyes can come from pigment, thin skin or the shadow of the eye socket. A phone camera cannot tell these apart, so Skinby says only that the skin is darker than usual and never says which cause it is.
What is usual comes from 763 photos from the web, grouped into 6 skin tones, with fewer for the darkest group. I will reset it with real phone photos from the beta.
These were added to photos on a computer, so they do not include real spot shapes, visible blood vessels, makeup or beards.
Stubble and beards can look like brown pigment, and the hollow under the eye casts a shadow that looks like it. I have not tested either, so Skinby never reports brown pigment on the chin or under the eyes.
A hard shadow across the face can still look like pigment. I treat pigment as the least certain reading and show it only when the light is even.
What is not measured yet
- Whether a bump is a whitehead, a blackhead or a cyst.
- Scars, and telling red marks from brown marks left by a breakout.
- Counting spots, or grading how severe acne is.
- Texture, dryness, oiliness and fine lines.
Skinby gives no score or grade for any of these yet.
Results from real use
There are none yet. When people have used Skinby long enough, those results will be shown here separately from the photo tests above.
Where the data comes from
The condition suggestions were trained on SCIN, a collection of skin photos with dermatologist labels that Google shares under an open licence (CC BY 4.0), plus other openly licensed photos credited in the app. Your own photos stay on your phone.