Styjun

I need to extract alt tags from images (only in body text) on a page. The code below fails to grab them all but just the first one on the page.

r = requests.get('https://www.bbc.co.uk/news/uk-politics-47648565')
soup = BeautifulSoup(r.content, "html.parser")

alt_tags = []
bio_img_soup = [s for s in soup.find_all( 'span', 'class': 'image-and-copyright-container')]
for div in bio_img_soup:
 for img in div.find_all('img', alt=True):
 alt_tags.append(img['alt']) 
print(alt_tags)

Could anyone direct me to a solution? Thanks!

UPD:

When using selenium, as suggested below, sometimes it works but sometimes it still grabs only the first image.

This is the code:

url = 'https://www.bbc.co.uk/news/uk-politics-47648565'

driver = webdriver.Chrome('/Users/vissea01/Downloads/chromedriver')
driver.get(url)

html = driver.page_source
soup = bs4.BeautifulSoup(html, "html.parser")

bios = []
bio_img_soup = [s for s in soup.find_all( 'span', 'class': 'image-and-copyright-container')]
for div in bio_img_soup:
 for img in div.find_all('img', alt=True):
 bios.append(img['alt'])
bios = [i for i in bios if i != 'Presentational grey line' and i != 'Presentational white space']
print(bios)

driver.close()

The same code outputs:

['Theresa May arriving in Brussels']

OR

['Theresa May arriving in Brussels', 'Analysis box by Katya Adler, Europe editor', 'Brexit timetable', 'Jeremy Corbyn']

the page is dynamic. when you do the request, that first image is part of the html source code. The other images are rendered after. You can use Selenium to render the page first, then pull all the img tags. You can use Selenium then to get those tags, or if you're like me and just feel more comfortable with bs4, you can use that.

from selenium import webdriver
from selenium.webdriver.common.by import By
import bs4
import pandas as pd

url = 'https://www.bbc.co.uk/news/uk-politics-47648565'

driver = webdriver.Chrome()
driver.get(url)

html = driver.page_source
soup = bs4.BeautifulSoup(html, "html.parser")

imgs = soup.find_all('img', alt=True)

for img in imgs:
 print (img['alt'])

driver.close()

Output:

Theresa May arriving in Brussels
Presentational grey line
Presentational grey line
Presentational grey line
Analysis box by Katya Adler, Europe editor
Presentational grey line
Brexit timetable
Presentational white space
Jeremy Corbyn
Theresa May arriving in Brussels
Anti-Brexit protests
Police at Parliament
‘It’s actually really good to get rejected’
How Brexit changed the English language
A forgotten food of the American South
Why water is one of the weirdest things in the Universe
What happens when we run out of food?
Canada's lake of methane
Imprints on the Sands of Time
Air India suspends Birmingham flights
Hen party mum to be buried in wedding dress
Is Kosovo’s capital city the ugliest in Europe?
Can a film be banned in the US?
Christine Chubbuck: The broadcaster who shot herself on air
[Gallery] The Worst Food From Every Single State
3 Ways Your Dog Asks For Help
[Gallery] This Is The Reason Clint Eastwood Never Discussed His Military Service
Seniors With No Life Insurance Feel Silly For Not Knowing This
No It's Not Oregano -- But This Plant Could Help You Retire Filthy Rich
This Holistic Remedy Improves Nail Fungus
Guns
Lauren and Dan Perkins with their six children
cyclone
Girl
Computer graphics
Guatemala village
Paris and Nanchanok
Kenyan boys and fishermen on Lake Victoria
Jacinda Ardern hugs woman
football being kicked on a field - Vauxhall image blurred in the background.