Datasets:
image_url
string
| user_id
string
| caption
string
|
---|---|---|
"http://static.flickr.com/2723/4385058960_b0f291553e.jpg" | "47889917@N08" | "A wooden chair in the living room" |
"http://static.flickr.com/1192/5148648301_1174ef59bc.jpg" | "29708551@N00" | "white horse near avebury" |
"http://static.flickr.com/4104/5079196943_c218abbf11.jpg" | "46339232@N07" | "Yellow flower surrounded by scorched black stalks - Moore Nature Reserve" |
"http://static.flickr.com/3286/2596097994_53cf4ffd61.jpg" | "69405057@N00" | "King Arthur's beheading rock - right on the sidewalk in the middle of town" |
"http://static.flickr.com/193/499952586_224e4f1712.jpg" | "79837140@N00" | "This is a shot of the Brittanic flag flying atop a farmhouse beside a field of megaliths" |
"http://static.flickr.com/4114/4740880938_60471e20ba.jpg" | "48341195@N04" | "It was taken when the season was running out and only this lonely flower was left in the field." |
"http://static.flickr.com/3176/3068205323_b373ea977c.jpg" | "32887925@N07" | "Coma sleeping on her bed in the front bedroom (2008)" |
"http://static.flickr.com/5056/5425290791_24ed1ece69.jpg" | "20144640@N00" | "Photos from a trip to a castle tower above Ljubljana." |
"http://static.flickr.com/5295/5429321291_e21b1ec057.jpg" | "32267947@N06" | "Frost in my bathroom window as seen on a cold winter day.Cabri, Saskatchewan in February 2011" |
"http://static.flickr.com/2802/4235682361_65ea851376.jpg" | "39882612@N04" | "this was the sun and some tree branches in front of my bedroom window.. i inverted and edited." |
"http://static.flickr.com/3122/2591505632_e6241f1ae7.jpg" | "8328184@N02" | "A bottle in the main door of Tiffany & Co." |
"http://static.flickr.com/2252/1993561876_213da63548.jpg" | "17904502@N00" | "wild ducks in a big sweet water lake in Guizhou, China" |
"http://static.flickr.com/4114/4862095670_483a0dcce0.jpg" | "43472658@N07" | "Asbestos is used willy-nilly in ukraine as building material. Dad and Tanya have an asbestos roof and an asbestos fence." |
"http://static.flickr.com/4063/4528077761_d10887932b.jpg" | "99746053@N00" | "Iris near the tri-foliate orange tree" |
"http://static.flickr.com/44/142417483_a5ce8fdd95.jpg" | "48078717@N00" | "View out the window of the upstairs bedroom of Mercedes' place in Holland, MI. August 2004." |
"http://static.flickr.com/3067/2836609074_6eb2637de4.jpg" | "56665574@N00" | "bridge street in the rain .." |
"http://static.flickr.com/1430/883779238_4f72a4ab39.jpg" | "52848143@N00" | "sitting in my computer chair." |
"http://static.flickr.com/4091/5056132211_610c4f746d.jpg" | "32762779@N07" | "You see these in orange, magenta, red and white all over Tenerife. A riot of colour." |
"http://static.flickr.com/1221/1425468692_de257d1fdf.jpg" | "9143581@N03" | "box elder bugs on one of my old 33" truck tires layin under the tree" |
"http://static.flickr.com/4151/5071418536_83dd3275f1.jpg" | "12544404@N06" | "Kellogs in a pizza box =) (c) celine jacinto 2010. all rights reserved." |
"http://static.flickr.com/1400/1436665871_2bb486d7b1.jpg" | "8435962@N06" | "A pink car on the ferris wheel in Odaiba." |
"http://static.flickr.com/1143/1189017424_dc0de2071d.jpg" | "9474298@N08" | "Me in a tree at my school in France! boy was I high up there!" |
"http://static.flickr.com/3564/3486923013_e486384592.jpg" | "7745738@N08" | "Area just north of hydro-electric plant near Great Fall SC. Power lines can be seen just above the tree line" |
"http://static.flickr.com/1126/5157409353_805483d0e4.jpg" | "32497732@N07" | "Dan, waiting for fish to jump in the boat so he can retake the lead from the NG" |
"http://static.flickr.com/101/303926874_7ccb2827ba.jpg" | "64749159@N00" | "Mommy, Daddy and Chase in a paddle boat on the lake at the Tarara Winery outside of Lucketts, VA." |
"http://static.flickr.com/2215/2398172845_6b387b69fe.jpg" | "16918641@N05" | "Monkeys up in the tree as seen from our river cruise along the Belize River." |
"http://static.flickr.com/2300/2526143599_e36bbabc2f.jpg" | "27607575@N00" | "Chicago at night from bridge over the river" |
"http://static.flickr.com/12/15946915_3c6a0d54ec.jpg" | "75881025@N00" | "bits of glass smoothed over by the waves and sand" |
"http://static.flickr.com/4083/5027176581_611ffc284f.jpg" | "53170558@N03" | "The day is over .. the big red sunball goes to sleep painting the sky in red" |
"http://static.flickr.com/3348/3183472382_156bcbc461.jpg" | "87547772@N00" | "The plate near the door almost makes it look like the CCTV camera is the logo of the hotel." |
"http://static.flickr.com/3374/3413419063_2c4187b431.jpg" | "44124423673@N01" | "A new church in Leam with cool window reflections." |
"http://static.flickr.com/2590/3710875431_cb058713a0.jpg" | "28100651@N07" | "A corner view of a brick building in black & white." |
"http://static.flickr.com/3448/3998667825_b9493a8432.jpg" | "42744010@N00" | "darren on the corner of beach blvd & main st. bsl bridge in the background. soooo tired of streets torn up." |
"http://static.flickr.com/3067/3044615132_60c54cf11a.jpg" | "22767894@N02" | "Brix sitting in her rocking chair on the front porch" |
"http://static.flickr.com/2667/4213264900_23ef5ba6b9.jpg" | "45813919@N07" | "This boat man has to pull the bamboo boat across every divider as the water does not flow across in low tide." |
"http://static.flickr.com/1358/715611858_067c52eba5.jpg" | "8188078@N04" | "This is a flower of a gourd plant in my backyard." |
"http://static.flickr.com/3134/2634673983_d61589f4cc.jpg" | "44831858@N00" | "A little bird built her nest underneath the roof of my veranda this spring. She hatched 4 babies." |
"http://static.flickr.com/170/421675368_05b29a68da.jpg" | "28217619@N00" | "Felix in his uber cute argyle cat sweater. Too bad he HATED it...with a passion." |
"http://static.flickr.com/114/291586170_82b2c80750.jpg" | "20794436@N00" | "A close look at winter stoneflies reveals mottled wings and black or brown bodies. Photo by Jason du Pont." |
"http://static.flickr.com/2512/4020118265_aaf6aaf805.jpg" | "79204206@N00" | "This crazy dog was jumping up and down at the tree and running all around" |
"http://static.flickr.com/4125/4996742390_078524ed56.jpg" | "46485566@N04" | "Out of my bedroom window at 8 o' clock in the evening" |
"http://static.flickr.com/3093/2700935643_a3d938cab0.jpg" | "42886798@N00" | "Had sleeping bags and pillows in the back of the truck and a chair for Bill." |
"http://static.flickr.com/2413/2345638099_664b4ba75f.jpg" | "23485843@N02" | "Sunset red sky over Koltur" |
"http://static.flickr.com/3488/4012278092_ddeb4a3024.jpg" | "80811313@N00" | "A train trellis over a body of water surrounded by trees turning." |
"http://static.flickr.com/129/417712961_cb09793ecd.jpg" | "82635623@N00" | "Bird Running in water on beach near Monterey" |
"http://static.flickr.com/3619/3637128249_6ae525f92a.jpg" | "35484044@N05" | "Cinnamon and nutmeg in tree form. Delicious! Your wall will thank you!" |
"http://static.flickr.com/5170/5208095391_f030f31dd1.jpg" | "22976502@N05" | "Karipol Leipzig, Germany, abandoned factory for house und car cleaning supplies in leipzig. founded in 1897, closed in 1995." |
"http://static.flickr.com/169/412851450_8d76daaeee.jpg" | "80789365@N00" | "The pump house (second building in the background) and some other building next to it (in the foreground)" |
"http://static.flickr.com/3227/2592867697_c735ce5df8.jpg" | "14090498@N04" | "The best picture of Maybree in this cake box was taken by Lorna or her mom with Lorna's camera." |
"http://static.flickr.com/3146/2505838070_79f5785960.jpg" | "15737284@N06" | "This box has two spearmint plants in it and a chocolate mint plant which smells really awesome." |
"http://static.flickr.com/3272/2820183980_6c9ff7eef4.jpg" | "35781259@N00" | "A cozy street cafe with a canopy and the colorful reflection in a modern glass building" |
"http://static.flickr.com/47/133865179_ac6b26064f.jpg" | "30607051@N00" | "fire boat at the dock's under the bay bridge just after dawn - embarcadero, san francisco, california" |
"http://static.flickr.com/2067/2244885255_5c67f094a3.jpg" | "66798295@N00" | "Cool tower at the top of the mountain in the rainforest" |
"http://static.flickr.com/70/201686190_baf3bc55dd.jpg" | "89612536@N00" | "not a cloud in the sky (marin trail)" |
"http://static.flickr.com/4144/5058259141_74056f7106.jpg" | "9186480@N05" | "Glass knobs in Stone Glass design for furniture knobs, dresser knobs, bathroom knobs, kitchen knobs and more. Custom sizes available by Uneek Glass Fusions." |
"http://static.flickr.com/3067/4554530891_f0d45b0967.jpg" | "33932135@N08" | "This train car is parked permanently in a yard in Royal, Nebraska." |
"http://static.flickr.com/4049/4435651563_34dd437685.jpg" | "35813754@N05" | "over the door shoe organizer" |
"http://static.flickr.com/2393/1509403365_a9a4ad5f6b.jpg" | "90554698@N00" | "mr. and mrs. wright and mr. and mrs. van der hyde. not a bird in sight...." |
"http://static.flickr.com/3058/3020749919_94fdd5511d.jpg" | "82088782@N00" | "Looking at the new bridge construction with the Kate Shelley Bridge in the background. There is a coal train going by." |
"http://static.flickr.com/1402/755073199_067a1a07e9.jpg" | "7255679@N04" | "some with glass, some only have a white glaze....these will have glass added at earthenware temperature. Some will have lustres added in a seperate firing" |
"http://static.flickr.com/3404/3339350856_2cf44be6e1.jpg" | "7399465@N06" | "The empty lot next door shot from computer room window. Usualy there are Mallards swimming in the temporary ponds." |
"http://static.flickr.com/4113/5087338290_4f47f9012f.jpg" | "24643141@N00" | "sand castle on Copacabana beach in Rio de Janeiro" |
"http://static.flickr.com/2748/4483200290_c2077fe8e5.jpg" | "33896471@N06" | "Pine tree and the sky above the San Juan National Forest in Colorado" |
"http://static.flickr.com/4014/4694972008_4cb1f324d6.jpg" | "17129636@N00" | "pedestrian over bridge with leaping girl" |
"http://static.flickr.com/2113/2685554482_265b096abe.jpg" | "57822052@N00" | "Camera is sitting on a hotel towel on the air-conditioner with the lense closed in the sliding window." |
"http://static.flickr.com/3323/3480602198_a00f6af835.jpg" | "21868470@N03" | "Tomato plant in the ground" |
"http://static.flickr.com/161/340024759_3c75d17140.jpg" | "92891603@N00" | "where they replaced the squishy floor in the second bathroom" |
"http://static.flickr.com/3097/2573168687_99dc54a245.jpg" | "21510583@N04" | "I found this old farm house at the base of the Alps near Castle Neuschwanstein." |
"http://static.flickr.com/3106/3220963258_4df41d4fda.jpg" | "8226961@N05" | "Minor road in Trivandrum, East Fort, viewed from roof top restaurant China Town" |
"http://static.flickr.com/114/286868463_9a78b87730.jpg" | "11888997@N00" | "Tunnel with the walking path near my house with trees" |
"http://static.flickr.com/3251/3053575098_b4b18116a7.jpg" | "60642416@N00" | "An alkaline and hypersaline lake in California, United States that is a critical nesting habitat for several bird species." |
"http://static.flickr.com/2587/3662526794_e5e65eb7af.jpg" | "39313606@N07" | "Somewhere in this house there's a magic mirror, a talking candle stick, and a glowing rose underneath a bell jar" |
"http://static.flickr.com/3145/2858049135_0944b40a34.jpg" | "29670082@N04" | "Another shot of the building next to the sixth floor museum in Dallas" |
"http://static.flickr.com/3551/4595980953_b13f619c38.jpg" | "8282860@N02" | "Dnieper river under dramatic sky" |
"http://static.flickr.com/3415/3431847916_9da0db1cc0.jpg" | "9902927@N05" | "Site 14 outside Tecate Mexico building a house by the TKA students." |
"http://static.flickr.com/1069/1317408378_15314e6bb1.jpg" | "12635387@N04" | "followed into the bathroom by a boy" |
"http://static.flickr.com/3101/2813975827_10ab7008c9.jpg" | "30147402@N08" | "A very surly looking black cat in a very hot Italian village." |
"http://static.flickr.com/3376/4566152547_77d8aefa02.jpg" | "99657969@N00" | "Mabel and Daisy enjoying the yummy green grass in the hayfield, while their summer home is being fenced off and built." |
"http://static.flickr.com/3379/3599112747_a3dc8a3c75.jpg" | "96822366@N00" | "Spider rock dominates the landscape at canyon de chelly in northern arizona" |
"http://static.flickr.com/2188/2382117129_2bd4c150d9.jpg" | "25254880@N07" | "Gorgeous wedding dress detailed in black and white." |
"http://static.flickr.com/45/134330497_c70aad2fc8.jpg" | "46438192@N00" | "found in a vending machine in a train station in switzerland" |
"http://static.flickr.com/4113/5212209590_6c7d40a7f6.jpg" | "22786607@N00" | "a tropical forest in the train station! how cool is that!" |
"http://static.flickr.com/2329/2222583275_b593822a7c.jpg" | "22702863@N06" | "kid at table 032307. todd in chair 053006." |
"http://static.flickr.com/3053/2857277803_17681e5c96.jpg" | "29818212@N03" | "Caurosel horse with Eiffel tower in background" |
"http://static.flickr.com/3259/3248336190_6fba7d8631.jpg" | "10950249@N07" | "Mallard duck in in the river Stour in Canterbury" |
"http://static.flickr.com/4142/4816011713_83ee950d87.jpg" | "51670120@N08" | "Yet another tree stump in the truck" |
"http://static.flickr.com/44/151667255_2103ab173b.jpg" | "40617004@N00" | "out of the oven and fruit mixed in" |
"http://static.flickr.com/24/48190716_c009cd19fb.jpg" | "59417938@N00" | "This is a glass ceiling in the lobby of an office building." |
"http://static.flickr.com/83/264002919_6932d04004.jpg" | "71362151@N00" | "Not the best idea to roll around on my floor wearing white" |
"http://static.flickr.com/5096/5390888578_ee4d403b21.jpg" | "58771608@N06" | "girl in the strange red sunglasses" |
"http://static.flickr.com/1338/4597496270_7d91f01f88.jpg" | "14914520@N06" | "le hast le hill in le pink le shirt" |
"http://static.flickr.com/36/94365550_6a69c36f3d.jpg" | "62697203@N00" | "fork in red velvet cake" |
"http://static.flickr.com/2176/1499775826_da81714393.jpg" | "14524113@N08" | "A view of the warning track through the fence in the left field bleachers--April 9, 2007" |
"http://static.flickr.com/3210/3140369358_72ca2cf96e.jpg" | "42744010@N00" | "new twin span bridge goign over lake pontchartrain" |
"http://static.flickr.com/5206/5242337131_02de94120e.jpg" | "53964109@N04" | "Accessible check-out window in a doctor's office waiting room." |
"http://static.flickr.com/5289/5311212162_4f90887615.jpg" | "17959488@N04" | "Conrail SD50 No. 6749 leads a westbound TV train approaching the Route 53 bridge near Cresson, PA." |
"http://static.flickr.com/5005/5301982645_0d1b80596e.jpg" | "11534815@N06" | "US mail box found in the middle of the street" |
"http://static.flickr.com/208/521783377_64a94c6aa8.jpg" | "74128918@N00" | "red sky over london" |
"http://static.flickr.com/2648/4154137062_8bd9dc7f35.jpg" | "42040949@N03" | "my sister's dog Diva in the car" |
"http://static.flickr.com/1192/1449179986_f2f725b443.jpg" | "12162840@N04" | "baby magic candle in blue" |
Dataset Card for SBU Captioned Photo Dataset
Dataset Summary
SBU Captioned Photo Dataset is a collection of associated captions and images from Flickr.
Dataset Preprocessing
This dataset doesn't download the images locally by default. Instead, it exposes URLs to the images. To fetch the images, use the following code:
from concurrent.futures import ThreadPoolExecutor
from functools import partial
import io
import urllib
import PIL.Image
from datasets import load_dataset
from datasets.utils.file_utils import get_datasets_user_agent
USER_AGENT = get_datasets_user_agent()
def fetch_single_image(image_url, timeout=None, retries=0):
for _ in range(retries + 1):
try:
request = urllib.request.Request(
image_url,
data=None,
headers={"user-agent": USER_AGENT},
)
with urllib.request.urlopen(request, timeout=timeout) as req:
image = PIL.Image.open(io.BytesIO(req.read()))
break
except Exception:
image = None
return image
def fetch_images(batch, num_threads, timeout=None, retries=0):
fetch_single_image_with_args = partial(fetch_single_image, timeout=timeout, retries=retries)
with ThreadPoolExecutor(max_workers=num_threads) as executor:
batch["image"] = list(executor.map(fetch_single_image_with_args, batch["image_url"]))
return batch
num_threads = 20
dset = load_dataset("sbu_captions")
dset = dset.map(fetch_images, batched=True, batch_size=100, fn_kwargs={"num_threads": num_threads})
Supported Tasks and Leaderboards
image-to-text
: This dataset can be used to train a model for Image Captioning where the goal is to predict a caption given the image.
Languages
All captions are in English.
Dataset Structure
Data Instances
Each instance in SBU Captioned Photo Dataset represents a single image with a caption and a user_id:
{
'img_url': 'http://static.flickr.com/2723/4385058960_b0f291553e.jpg',
'user_id': '47889917@N08',
'caption': 'A wooden chair in the living room'
}
Data Fields
image_url
: Static URL for downloading the image associated with the post.caption
: Textual description of the image.user_id
: Author of caption.
Data Splits
All the data is contained in training split. The training set has 1M instances.
Dataset Creation
Curation Rationale
From the paper:
One contribution is our technique for the automatic collection of this new dataset – performing a huge number of Flickr queries and then filtering the noisy results down to 1 million images with associated visually relevant captions. Such a collection allows us to approach the extremely challenging problem of description generation using relatively simple non-parametric methods and produces surprisingly effective results.
Source Data
The source images come from Flickr.
Initial Data Collection and Normalization
One key contribution of our paper is a novel web-scale database of photographs with associated descriptive text. To enable effective captioning of novel images, this database must be good in two ways: 1) It must be large so that image based matches to a query are reasonably similar, 2) The captions associated with the data base photographs must be visually relevant so that transferring captions between pictures is useful. To achieve the first requirement we query Flickr using a huge number of pairs of query terms (objects, attributes, actions, stuff, and scenes). This produces a very large, but noisy initial set of photographs with associated text.
Who are the source language producers?
The Flickr users.
Annotations
Annotation process
Text descriptions associated with the images are inherited as annotations/captions.
Who are the annotators?
The Flickr users.
Personal and Sensitive Information
Considerations for Using the Data
Social Impact of Dataset
Discussion of Biases
Other Known Limitations
Additional Information
Dataset Curators
Vicente Ordonez, Girish Kulkarni and Tamara L. Berg.
Licensing Information
Not specified.
Citation Information
@inproceedings{NIPS2011_5dd9db5e,
author = {Ordonez, Vicente and Kulkarni, Girish and Berg, Tamara},
booktitle = {Advances in Neural Information Processing Systems},
editor = {J. Shawe-Taylor and R. Zemel and P. Bartlett and F. Pereira and K.Q. Weinberger},
pages = {},
publisher = {Curran Associates, Inc.},
title = {Im2Text: Describing Images Using 1 Million Captioned Photographs},
url = {https://proceedings.neurips.cc/paper/2011/file/5dd9db5e033da9c6fb5ba83c7a7ebea9-Paper.pdf},
volume = {24},
year = {2011}
}
Contributions
Thanks to @thomasw21 for adding this dataset
- Downloads last month
- 186
Data Sourcing report
No elements in this dataset have been identified as either opted-out, or opted-in, by their creator.