Program that pulls ALL links from a website
Required skills: C/C++, Excel, Ruby/Ruby on Rails, Visual Basic, Web Scraping
I am in need of a simple script/program that will copy ALL internal and external links from a website.
The script must not copy relative links, they should all be full links that could be copied into an address bar and work. (therefore it also should not abbreviate or shorten the links with … or any other foreign text)
Other criteria:
-Links to be output into an Excel file named after the domain of the website (.xlsx preferred but other Excel formats acceptable)
-Script/program must be highly reliable. Many websites have upwards of 10,000 to 50,000 internal and external links (however I will NOT be copying giants like Wikipedia or ebay)
-Due to the nature of redirecting links, redirects are not expected to copy the link being redirected
-If the Excel file can be automatically filtered of duplicates and sorted by name (ie column A sorted A>Z) I would like that. However if this is a significant burden on the script, I DO NOT want this because I do not want any possible variables that could slow the script or cause it to stop midway through a link scrape.
If you can create a script that is reasonably user friendly please post your bid.
The script must not copy relative links, they should all be full links that could be copied into an address bar and work. (therefore it also should not abbreviate or shorten the links with … or any other foreign text)
Other criteria:
-Links to be output into an Excel file named after the domain of the website (.xlsx preferred but other Excel formats acceptable)
-Script/program must be highly reliable. Many websites have upwards of 10,000 to 50,000 internal and external links (however I will NOT be copying giants like Wikipedia or ebay)
-Due to the nature of redirecting links, redirects are not expected to copy the link being redirected
-If the Excel file can be automatically filtered of duplicates and sorted by name (ie column A sorted A>Z) I would like that. However if this is a significant burden on the script, I DO NOT want this because I do not want any possible variables that could slow the script or cause it to stop midway through a link scrape.
If you can create a script that is reasonably user friendly please post your bid.
Posted In:Visual BasicC/C++ExcelRuby/Ruby on RailsWeb Scraping |
Related projects:
Extract all products from a website
URL
I will need to see it demonstrated and tested out on my system to make sure that it fully works without any errors before payment for the work is finalized.
Will provide the Site Url via PM to interested providers.
Please send us an eg of a Site that you have extracted data from and a example excell file .
Need work to be done properly and quickly. escrow payment.
I will need to see it demonstrated and tested out on my system to make sure that it fully works without any errors before payment for the work is finalized.
Will provide the Site Url via PM to interested providers.
Please send us an eg of a Site that you have extracted data from and a example excell file .
Need work to be done properly and quickly. escrow payment.
I need a program that can recognize numbers from a form.
ted in, and they appear a few times a day, but sitting there keeping an eye on it all the time is not possible :P
So i want a program that could play a loud sound or something when a surebet over 9 or 10 % appears.
I will show pictures that have relevance to the project in PM's or mails only, not here public, because the pics is from a paysite, and i dont wanna do commercial.
Thx.
So i want a program that could play a loud sound or something when a surebet over 9 or 10 % appears.
I will show pictures that have relevance to the project in PM's or mails only, not here public, because the pics is from a paysite, and i dont wanna do commercial.
Thx.
scrape ALL LINKS in an website - java applet
don't want any prompt for visitors) to scrape ALL LINKS on a website. All Links mean even links in iframes (facebook widget and ads).
This applet will be place on the website that i want to scrape and must display links.
Before choosing a bidder i need absolutely to test a demo version.
This applet will be place on the website that i want to scrape and must display links.
Before choosing a bidder i need absolutely to test a demo version.
Program to export data from a website to CSV excel
is an auction like website: there's a list of items, and for each one I'd like to extract information such as winning bid, date, number of bids, etc.
Further information will be given at the next stage...
Attention!!
1. Bidders must know GOOD English.
2. I'm willing to pay no more than $ 30, so don't bother bidding if you're price is higher than that.
3. Quick delivery time
Further information will be given at the next stage...
Attention!!
1. Bidders must know GOOD English.
2. I'm willing to pay no more than $ 30, so don't bother bidding if you're price is higher than that.
3. Quick delivery time
List All External Links From A Website
We need a PHP website crawler written that spiders a website listing all of the external (outbound) links in the website.
Each listing needs to contain all the attributes of the link including, URL, Anchor Text, no-follow, image, etc, etc.
All of the results need to be written to a MYSQL database and be emailed to a predetermined email address with a CSV attachment of the report.
Each listing needs to contain all the attributes of the link including, URL, Anchor Text, no-follow, image, etc, etc.
All of the results need to be written to a MYSQL database and be emailed to a predetermined email address with a CSV attachment of the report.
I need a program to collect links from a website
I need a program that can crawl through this entire site and collect links. It needs to have a PR option. Example. If I specify PR 4 - it will collect all pr4 and above. If I say pr0 - it will collect pr0 and above.
Please let me know if this can be done.
Thank you!
The site is
http://www.web-directories.ws/sitemap.php
Please let me know if this can be done.
Thank you!
The site is
http://www.web-directories.ws/sitemap.php
Get the data from a website and put in a program vb6
tached doc's.Project budget is 300 american dollar's please check that before you bid!
Additional files submitted:
detailes.txt
Additional files submitted:
detailes.txt
Harvest Links From A Website
Hello,
I'd like someone to harvest the website links from this directory
http://unionist.com/every-union-website
The file should be an xls file with a column for the websites
Project turnaround should be 24 hours.
Thank you.
I'd like someone to harvest the website links from this directory
http://unionist.com/every-union-website
The file should be an xls file with a column for the websites
Project turnaround should be 24 hours.
Thank you.
Data Entry from a website to Excel
me Phone Fax Contact Email Website Biz Description
Therefore there are just 4 pieces of Information per company which needs to be filled in.
As I mentioned above, this is not difficult work, just time consuming. I would estimate that this should take about 8-10 hours to complete.
The information added must be correct and therefore I need someone who has a keen eye for detail.
If interested please post a bid.
Therefore there are just 4 pieces of Information per company which needs to be filled in.
As I mentioned above, this is not difficult work, just time consuming. I would estimate that this should take about 8-10 hours to complete.
The information added must be correct and therefore I need someone who has a keen eye for detail.
If interested please post a bid.
Help Find Private Links Of A Website
re several 'worries' about this method and we'd have to talk about this in more detail.
I'm sure there are other ways that I am not aware of, if you can find the links using other methods that would be fine too. The job is to just get the links. I have already looked in wayback archive and found a few of these links, so I can provide you with the format the URLs are usually set in.
We'd have to chat about this before the 'job' to see if this is feasible.
Thanks
I'm sure there are other ways that I am not aware of, if you can find the links using other methods that would be fine too. The job is to just get the links. I have already looked in wayback archive and found a few of these links, so I can provide you with the format the URLs are usually set in.
We'd have to chat about this before the 'job' to see if this is feasible.
Thanks
Data Entry from a website to Excel
Email Website Biz Description
Therefore there are just 4 pieces of Information per company which needs to be filled in.
As I mentioned above, this is not difficult work, just time consuming. I would estimate that this should take about 8-10 hours to complete.
The information added must be correct and therefore I need someone who has a keen eye for detail.
If interested please post a bid.
Mark
Therefore there are just 4 pieces of Information per company which needs to be filled in.
As I mentioned above, this is not difficult work, just time consuming. I would estimate that this should take about 8-10 hours to complete.
The information added must be correct and therefore I need someone who has a keen eye for detail.
If interested please post a bid.
Mark
data scraping from a website to XLS
br>5 th column: all subcategories
6 th column: Category product
7 th column: product name
8th column: description of the complete article
9th column: item number
10th column: links to the photos
11 th Column : the price
I can send you an exampel result in XLS
Only serious People pls react
Requirement
you have experience in data scraping
rigorously in the work
thank you
6 th column: Category product
7 th column: product name
8th column: description of the complete article
9th column: item number
10th column: links to the photos
11 th Column : the price
I can send you an exampel result in XLS
Only serious People pls react
Requirement
you have experience in data scraping
rigorously in the work
thank you
data scraping from a website to XLS
r>6 th column: Category product
7 th column: product name
8th column: description of the complete article
9th column: item number
10th column: links to the photos
11 th Column : the price
I can send you an exampel result in XLS
Only serious People pls react
Requirement
you have experience in data scraping
rigorously in the work
**** Serious about Deadline ****
thank you
7 th column: product name
8th column: description of the complete article
9th column: item number
10th column: links to the photos
11 th Column : the price
I can send you an exampel result in XLS
Only serious People pls react
Requirement
you have experience in data scraping
rigorously in the work
**** Serious about Deadline ****
thank you
Data Collection from a website
3 digit code for all the pdf's on the site, but currently it is taking too long to guess codes to download the pdf's.
I need someone to get all the pdf's off the site and send them to me..
If the pdf's have been uploaded on the site in a date format that can be followed I also then want the new pdf's every month they upload more after that..
I need someone to get all the pdf's off the site and send them to me..
If the pdf's have been uploaded on the site in a date format that can be followed I also then want the new pdf's every month they upload more after that..
copy address from a website
entered into a spread sheet that I will provide. All address may not contain duplicates. A few would be understandable. All address will be checked for verivication that they came from this website. The website shows the address as public information so it is easily accesable. I will pay .10 per address that is entered into the spread sheet. You can pull as many as 1000 addresss if you want to .
Need program that creates reports from MS SQL DB
Looking for a tool that will create a web interface that pulls reports reports from a SQL server DB. The program should create an entire web application and should be easy enough for anyone to use.
Extraction of data from a website
number, fax number, e-mail address, website, VAT ID.
We would need the data sorted in columns in an excel file (.xls or .xlsx).
Please note that PIRS (www.pirs.si/eng) is a Business Directory of the Republic of Slovenia and it contains publicly available information. We need the information in order to accelerate our gathering of data from the abovementioned Business Directory (which is done by hand at the moment).
Thank you very much in advance!
We would need the data sorted in columns in an excel file (.xls or .xlsx).
Please note that PIRS (www.pirs.si/eng) is a Business Directory of the Republic of Slovenia and it contains publicly available information. We need the information in order to accelerate our gathering of data from the abovementioned Business Directory (which is done by hand at the moment).
Thank you very much in advance!
A Program That Imported A List of 10,000 activation Urls to automatically activate on the background!
am that will Import A List Of URL that start with http:// and https:// links. The program will import these links from a csv file and it will automatically activate these links in the background. It should activate these links fairly fast.
If you can do this project, more project will be coming for you in the future.
This is an URGENT PROJECT! Should be completed in 1 day. Very Easy Project. Whoever got fast response will most likely to get the job!
If you can do this project, more project will be coming for you in the future.
This is an URGENT PROJECT! Should be completed in 1 day. Very Easy Project. Whoever got fast response will most likely to get the job!
100 One Way Theme Links for a Website
issions (must be manually submitted)
-No redirects or cloaking (e.g., 301, 302, meta-refresh, JavaScript redirects)
-Links should not be excluded by a tag
-All links and link pages will be verified and checked before payment
***
After you have placed our link, you would need to send a report in excel format with the site's URL, site PR.
**Payment will definitely be made once all links are verified and meet the specified conditions above
-No redirects or cloaking (e.g., 301, 302, meta-refresh, JavaScript redirects)
-Links should not be excluded by a tag
-All links and link pages will be verified and checked before payment
***
After you have placed our link, you would need to send a report in excel format with the site's URL, site PR.
**Payment will definitely be made once all links are verified and meet the specified conditions above
Need data from a website form inserted into a Word .doc by undergroundfx
ata from a website form automatically inserted into individual Word .docs at different locations. Is this possible? Also this is on a Windows server using ASP. The rest of the site is taken care of and this is all that needs to be done.
It will be a link on and similar to the current questionnaire on www.auroraheadache.com . What I will provide is the questions to be made into a form and the word .doc with the blank spots in it where the data will go. Thanks.
It will be a link on and similar to the current questionnaire on www.auroraheadache.com . What I will provide is the questions to be made into a form and the word .doc with the blank spots in it where the data will go. Thanks.
