Ciugk

Clash Royale CLAN TAG #URR8PPP beautifulsoup for looping and getting text and Href Im in a bit of a quinch here: its an ASP site which is really messy that I am trying to get data from: Im trying to use a FOR loop to get an href and the text of all the rows of the 4th table that is on the site, so I first did: table = soup.findAll('table')[3] Then from this table I need to get all text inside the tags and the href's of the inside. i tried something like this: for product in table.findAll('tbody'): product_title = product.find('tr').text product_link = product.find('a')['href'] print (product_title, product_link) But I get nothing in return... :( Can you show a print of product ? – Rakesh 7 mins ago product What does your HTML document look like? –...

Clash Royale CLAN TAG #URR8PPP selenium firefox python won't locate upload button I'm using Selenium with Firefox (I've tried chrome but the success ratio on this one was significantly lower). I'm trying to locate an element (upload button) and the success ratio is nowhere near an acceptable percentage. Now I know that I could try to use just selenium to the input but it wont locate it as well no matter what I do. So I used AutoIt for the uploading part. frdriver.get('') time.sleep(11) try: # el = WebDriverWait(frdriver, 15).until(EC.presence_of_element_located((By.XPATH, '/html/body/div[1]/header/div[2]/div[2]/div/div/form/i[2]'))) el = frdriver.find_element_by_xpath('/html/body/div[1]/header/div[2]/div[2]/div/div/form/i[2]') try: el.click() print(el) #ActionChains(frdriver).move_to_element(el).click().perform() try: #element = WebDriverWait(frdriver, 15).until(EC.presence_of_element_locat...

Clash Royale CLAN TAG #URR8PPP BeautifulSoup4 IndexError: list index out of range? I am learning about BS4 and I can't figure out what this code does and why does this code throws out an error like this: Traceback (most recent call last): File "/home/ubuntu/workspace/Untitled1.py", line 17, in <module> for tr in soup.find_all('tr')[3]: IndexError: list index out of range The python code using bs4 is: for tr in soup.find_all('tr')[2]: tds = tr.find_all('td') print("value:%s,value 2:%s,value3:%s" (tds[0].text,tds[1].text,tds[2].text)) Thank you so much for reading this post. Does soup.find_all('tr') return list with equal to /greater than 3 values ? – Madhan M 3 mins ago equal to /greater than ...

Clash Royale CLAN TAG #URR8PPP How do I integrate a web scraper into a working website? I'm working on a project that requires web scraping to retrieve data from a website. I want to create a website that allows the user to input data, like a few keywords, and have a scraper scrape data relating to that keyword and then display that data on the website. I've gotten the scraper part down, but now I'm not sure how to make that into a website. Does that mean I'd have to integrate Python (web scraping lang) into HTML? I have to install packages to scrape too (like BeautifulSoup), will this impact the feasibility of the project? Overall, is this possible, and if not, what alternatives are there? Thanks in advance. By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to...

Clash Royale CLAN TAG #URR8PPP Can't figure out Beautifulsoup find() command for this HTML I am trying to scrape some info from a page with python and Beautiful soup and i cant seem to write the right path to what i need, the html is: <div class="operator active" data-operator_name="Etisalat" data- operator_id="5"><div class="operator_name_etisalat"></div></div> And i am trying to get that operator name "Etisalat", i got this far: def list_contries(): select = Select(driver.find_element_by_id('international_country')) select.select_by_visible_text('France') request = requests.get("https://mobilerecharge.com/buy/mobile_recharge?country=Afghanistan&operator=Etisalat") content = request.content soup = BeautifulSoup(content, "html.parser") # print(soup.prettify()) prov=soup.find("div", {"class": "operator active"})['data-operator_name']...

Clash Royale CLAN TAG #URR8PPP How can I download full webpage by a Python program? Currently I have a program that can only download the HTML of a given page. Now I want a program that can download all the files of the web page including HTML, CSS, JS and image files(Same as we get on ctrl-s of any website). My current program is: import urllib urllib.urlretrieve ("https://en.wikipedia.org/wiki/Python_%28programming_language%29", "t3.html") I have visited many such questions in Stack Overflow, but they are all only downloading the HTML file. So you want to go through the links in the HTML and download the content they point to? Note that a Wikipedia page contains links to other pages; do you want to do that recursively? – jonrsharpe Jul 3 '15 at 11:21 Yes i want to download all the link...