Website Crawler Application

WILL be held to it.

We will be adding A LOT of features to this crawler as time progresses so you will have A LOT of future work with us if you create this correctly, quickly and at a decent price.

We reserve the sole rights to this script and it cannot be re-purposed or resold to anyone else.

You must send me a PM with any comments, questions and/or your experience and reference this project so I can clearly see which bids are fake and which are real.

get="extlink" rel="nofollow" style="text-decoration: none;color:#68778E" onmouseover="this.style.textDecoration='underline'" onmouseout="this.style.textDecoration='none'">Moneybookers.com


CONTACT ONLY IF YOU ACCEPT ABOVE MENTIONED TERMS

---

Additional Info (Added 1/8/2010 at 23:16 EST)...System Message: This is a reposting of project Website + iPhone Application (1262343740).

I need a team/company to design and develop a website + iphone application.
Please contact me via PMB for more details.

Thank you.

ll provide the website url). The information we need from the website would be pertaining to inventory specifications such as (but not limited to):
Availability, Price, Image, Name, Description, Weight, etc…
Basically this application should be able to pull out an inventory list. This list should then either be stores in a flat file or database table. The application should be able to run whenever we want and generate a inventory list.
Preferably using php.

I need a crawler application that crawls one website ( I will provide URL). The application should be able to crawl specific url's of that website by keywords that I input, and do a general crawl as well. It should output the data (four fields) to an excel file.

must change IP addresses to avoid blocking (preferably every 10-20 searches )
Some airlines require account number and pwd to access availability(AIr France for example,). I want to be able to add more accounts in the future to a crawler so it will use random account and random IP to avoid blocking. I will provide you all account information if needed.
Please bid only if you have previous experience on similar scraping projects

sults for each of these URLS.

------------------------------------------
Testing

One at a time URL testing can be done online using W3C online tools:
http://validator.w3.org/
http://validator.w3.org/checklink/
http://jigsaw.w3.org/css-validator/


------------------------------------------
Follow-on work

There is an opportunity for further work.

r as time progresses so you will have A LOT of future work with us if you create this correctly, quickly and at a decent price.

We reserve the sole rights to this script and it cannot be re-purposed or resold to anyone else.

You must send me a PM with any comments, questions and/or your experience and reference this project so I can clearly see which bids are fake and which are real.

We need a website crawler script that will extract email addresses. It will be for one site only, but would be nice if it worked on others as well. This is a forum with 160,000 threads that need to be crawled, and all email addresses extracted.

Will need to run on a mac, so Python might be best unless you have a better idea.

Need this done ASAP - completed within the next few days.

78E" onmouseover="this.style.textDecoration='underline'" onmouseout="this.style.textDecoration='none'">http://www.werkspot.nl/company.php
Here are several cathegories listed under where all companies are going to be vissible.
Each company had a button "meer informatie" Under this button all the company information is displayed.
Whe need a simple crawler then can get the listen information from these pages.

Duplicates must be able to be removed automaticly.

tabase.
2. Pickup links from sites and index those.
3. Never stop unless it's out of links to index.
4. If the site has no meta tags, take content from the page.
5. copy some content from each page ( like 150 words ) and place that in the database...
6. Will allow me to change a setting about weather or not a site can be scanned twice. ( A Check Box )

I'm guessing roughly 60 lines of code. I'd love to have a pause button, but it's not necessary.

nt devices / computers / etc.... All of the computers that we use are Windows XP, and the website that these would be accessing is running linux, and php.

Hopefully this is enough info to get an idea of what I am looking for. Let me know if you need any clarification. I don't have a good idea of what the cost of making something like this would run (i'm in the web dev business), so if I'm way off, please comment in what it would actually take to get this done.

n a Linux server and be capable of being
scheduled to automatically run at certain times.

7. Admin functions must be allowed to enable the addition/removal of
sites and to update layout instructions per site.

Please reply with specific experience and examples of crawlers/spiders you have developed that meet similar requirements.

PLEASE NOTE : Generic replies with long lists of irrelevant sites will be DELETED automatically.

e;color:#68778E" onmouseover="this.style.textDecoration='underline'" onmouseout="this.style.textDecoration='none'">https://www.racewayautomotive.com/luxuryImports/creditapp.html

http://www.auctionadmin.com/CreditApp.aspx?iID=3172622

Contacts/Friends which will send out their vCard URL address to that person along with a link for them to sign-up for the service and create their own card. Recipients do NOT have to sign up in order to see the vCard information.

There is additional information in the attached file.

Please provide a quote for the business card and iPhone application combined, a timeframe for completion, and examples of your previous work.

events.
Robust reporting systems to track what is being done where & when.
and have the ability of being language enabled.

Please respond with work you have actually completed and you may pitch for the app or the website but preference will be for people who can undertake both. Please outline a rough process plan

The graphic design element is optional as i have a company that can do this if nessisary.

Regards

e a website similar as function like beemp3.com.
the website application also needs a crawler for mp3 files, those files needs to be read(artist name, song name, album, year) and stored into the database.
each song needs to have its own page+tags(artist name, song etc)+friendly url to be very fast indexed.
i need sitemap+module for ads by rotation.

im very intersted regarding the full functionality of website like this.

i,

We are an application and software development company.

We are looking for a website with a front end and back end connected to an Apple and Android application. This solution is for a copywriting service for email communications.

The project will be reoccurring and is a good start to form a lasting relationship. We started with a simple project pricing. If you have an inquiries please feel free to contact us.

ther half of the module developments is completed.
e. 15% when documentation and testing is completed.
f. 15% After 60 days from launching.

NOTE
This project has a time constraint, please don’t bid if you would assign it to juniors or may not have the necessary resources and skills.

t style="padding-left: 20px; width: 320px">


Additional information submitted: 07/22/2011 at 2:02 EDT:
Thanks,
James