ERLANG large scale concurrent real time website scanning
Required skills: Erlang
I'm looking for expert consulting on the feasibility (costs in terms of hardware/development) of scanning several million webpages in real-time or close to real time). Longer term employment may result if you can provide a solution that is feasible and within my budget. If you'd like this, you'd need to write a small mini-program to demonstrate the feasibilty and speed of your proposed solution. Knowledge of Python and mysql are a plus, but not required. This project may close as soon as I find someone suitable (it may not stay open the full 60 days).
Currently we have about several million html (no-rss feed) websites to scan stored in a database. I'd like to have them scanned on a real-time basis or a nearly real-time basis because the market data needs to be updated as fast as possible. I'm wondering if Erlang can do this.
Currently, these websites are stored in a database and scanned via brute force methods using approximately 1,000 IP's and three servers with xeon hex core's on each. These are all connected to a database server on the same network. With this, the scanning speed is approximatly 1-3 times every 1-2 hours depending on server load. This is too slow for our needs.
Currently we have about several million html (no-rss feed) websites to scan stored in a database. I'd like to have them scanned on a real-time basis or a nearly real-time basis because the market data needs to be updated as fast as possible. I'm wondering if Erlang can do this.
Currently, these websites are stored in a database and scanned via brute force methods using approximately 1,000 IP's and three servers with xeon hex core's on each. These are all connected to a database server on the same network. With this, the scanning speed is approximatly 1-3 times every 1-2 hours depending on server load. This is too slow for our needs.
Posted In:Erlang |
Related projects:
Real Time Crawler Website
wls selected auto classifieds websites. It should be working on the same principle as the usual travel, rent-a car, jobs crawling websites, but it will crawl for autos in real time.
So when somebody comes to my site, he should choose what vehicle, model, year, mileage, fuel etc. to search, then the site should return the related results from the sites pointed to.
for example look for car on my site, which will give you the results from mobile.de and autoscout24.de
So when somebody comes to my site, he should choose what vehicle, model, year, mileage, fuel etc. to search, then the site should return the related results from the sites pointed to.
for example look for car on my site, which will give you the results from mobile.de and autoscout24.de
Real-time Development
ut it needs to look more like www.foursquare.com
I need any status updates or changes to a product on this website to be brought into the timeline.
Basically please scan how buzzmart, and Foursquare work, and if you feel comfortable, place a bid. Thanks
I need any status updates or changes to a product on this website to be brought into the timeline.
Basically please scan how buzzmart, and Foursquare work, and if you feel comfortable, place a bid. Thanks
forex real time website
HI I WANT TO MAKE FOREX REAL TIME WEB SITE FOR LIVE NEWS
LIVE SIGNAL USING MY DESKTOP APPLICATION SOFTWARE META TRADER 4
FOR MORE DETAIL CONTACT ME
LIVE SIGNAL USING MY DESKTOP APPLICATION SOFTWARE META TRADER 4
FOR MORE DETAIL CONTACT ME
Penny Auction With Real Time Communications
time reset
* Auction time extend
* Affiliate Program (PostAffiliatePro)
* Auto Billing System (UK law)
* Email notification/verification
* Support Ticket Sistem
* Feedback system
* Programmable Newsletters/notifications
* SEO Optimization "friendly Url, Tag per every items"
* Auto Site Map
* Complete internal stats
* Social network application
Before choosing a candidate, we want to see test efficiency and reliability.
* Auction time extend
* Affiliate Program (PostAffiliatePro)
* Auto Billing System (UK law)
* Email notification/verification
* Support Ticket Sistem
* Feedback system
* Programmable Newsletters/notifications
* SEO Optimization "friendly Url, Tag per every items"
* Auto Site Map
* Complete internal stats
* Social network application
Before choosing a candidate, we want to see test efficiency and reliability.
Front End Design For Real Estate Website
E7, IE8, Firefox, Chrome, Safari and Opera.
Pages for which a HTML/CSS mock-up is required
- Portal like frontpage which allows quick access to all the functionality
- RE search result page listing one or more results
- RE product page which lists a picture of a house, basic properties etc
- Contact page to contact the owner/real estate agent, bid on a listed property, request more information etc.
- Site info page template, used for FAQ, policy, etc
Pages for which a HTML/CSS mock-up is required
- Portal like frontpage which allows quick access to all the functionality
- RE search result page listing one or more results
- RE product page which lists a picture of a house, basic properties etc
- Contact page to contact the owner/real estate agent, bid on a listed property, request more information etc.
- Site info page template, used for FAQ, policy, etc
Real Estate Website
cal extensions of the Real Estate site in item one.
The successful bidder will supply me with the url of at least one Real Estate website they have built in the last two years.
DO NOT simply send me a link to your portfolio. I will delete it without looking. Send me links to R. E. sites you have built only and I will look at your portfolio if I am impressed by the site(s).
Preference will be given to those having done this on a WP platform
Thank you
The successful bidder will supply me with the url of at least one Real Estate website they have built in the last two years.
DO NOT simply send me a link to your portfolio. I will delete it without looking. Send me links to R. E. sites you have built only and I will look at your portfolio if I am impressed by the site(s).
Preference will be given to those having done this on a WP platform
Thank you
Website - Real Time Statistics Code - Script Installation
Detailed script code update to existing website... I will provide HTML code from godaddy. This go allows for real time web statistics... Additionally, we will have three pictures to update our website, two online forms for customer to download and printer or fill out via PDF... Actually, both forms I can/will provide in word and PDF. They will also be 5-10 labels/word updates e.g. Letter of Agency instead of Energy 101 label. These should be simple.
Real Estate Website with RETS
we need from you to help us setting up RETS to transfer data from mls server to our server and convert that data into MySQL, after that if u really expert into PHP/Ajax, we will create CMS to manage the website
basically u are just bidding to help us to configure our RETS server (you might need to create us custom script which will transfer data from mls servers to our server)
any question, PM me
happy bidding...
basically u are just bidding to help us to configure our RETS server (you might need to create us custom script which will transfer data from mls servers to our server)
any question, PM me
happy bidding...
Real Estate Website Experienced Developer Needed
integration with MLS, MLX, IDX etc
- Able to create search setup and website look similar to this website http://www.gellens.com/
- Able to show how to update the listings on a daily basis
- Need to be able to design the website as well.
- Would like a content management system like wordpress to be used, but not necessary.
please provide Skype info, real estate site that you have done, include PHP experience as well
- Able to create search setup and website look similar to this website http://www.gellens.com/
- Able to show how to update the listings on a daily basis
- Need to be able to design the website as well.
- Would like a content management system like wordpress to be used, but not necessary.
please provide Skype info, real estate site that you have done, include PHP experience as well
New Large Scale Networking Website
mer must be willing to work under a confidentiality agreement.
Possibilities for long term work as the site will be constantly developed. Possible share in a Company with vast potential, for a top quality candidate that proves indispensible.
Quoted prices are not necessarily needed at this stage, an overview of expertise is essential.
Young, inexperienced, but very highly skilled candidates will also be considered.
Possibilities for long term work as the site will be constantly developed. Possible share in a Company with vast potential, for a top quality candidate that proves indispensible.
Quoted prices are not necessarily needed at this stage, an overview of expertise is essential.
Young, inexperienced, but very highly skilled candidates will also be considered.
Need Ready Website (Real Estate)
. I don't want any bids for developing these website. I will only entertain developers who have the projects ready. The website should have all the features of 99acres.com, magicbricks.com.
Make Sure the Phrase "Ready Made Projects" is mentioned in the bid to confirm that you have read the specifications properly.
Regards
Chandra Baid
BSquarre Services
chandra dot baid at rediffmail dot com
Make Sure the Phrase "Ready Made Projects" is mentioned in the bid to confirm that you have read the specifications properly.
Regards
Chandra Baid
BSquarre Services
chandra dot baid at rediffmail dot com
Real Estate Website
to update.
I will personally come up with the layout for the website. I would like it to be something very similar to this:
www.balistings.com
If you have any questions, please ask and I will be sure to answer them.
Serious inquires only and native English is preferred.
I will personally come up with the layout for the website. I would like it to be something very similar to this:
www.balistings.com
If you have any questions, please ask and I will be sure to answer them.
Serious inquires only and native English is preferred.
Large Scale Web Development
g on work speed and capabilities of developer product development could be quicker.
Individual would need to be willing to sign a NDA for the idea of the site as well as the site itself.
Will take all competitive bids.
I care about experience, creativity, and willingness to go the extra mile to make this website extraordinary.
Individual would need to be willing to sign a NDA for the idea of the site as well as the site itself.
Will take all competitive bids.
I care about experience, creativity, and willingness to go the extra mile to make this website extraordinary.
Set Up A Studio Press Real Estate Website
I need a Studio Press Real Estate Website set up. It needs to have a rental availability calendar on it - it is for a client, it does not need a huge amount of input it is a simple job. We have hosting, we have the domain as well
Mortgage And Real Estate Website Needed
that can work fast, start now, and most have great communication. Attached are the exact instructions.
To have a quick idea of what I am looking for. An Example of what I am looking for can be found at;
www.premierestateproperties.com
To have a quick idea of what I am looking for. An Example of what I am looking for can be found at;
www.premierestateproperties.com
Dynamic Real Estate Website
tomatically.
5. Manage and create events that end-users can RSVP too.
Junior Admin related functions
6. Upload all videos for training and sales
Can do any of the 4 points described above that the Senior Admin provides
access too.
Guest related functions
Can view the open area content and request access to restricted area
content
5. Manage and create events that end-users can RSVP too.
Junior Admin related functions
6. Upload all videos for training and sales
Can do any of the 4 points described above that the Senior Admin provides
access too.
Guest related functions
Can view the open area content and request access to restricted area
content
Co browsing feature - Real Time
Real Time cobrowsing feature needed in website.
Advanced Real-Time Chat Application/Website
nd leave groups.
Room specific groups and site wide groups.
Ability for users to create and manage new groups.
Main html/css layout
Future requirements:
Ability to integrate with Twitter accounts and hashtags
Administration panel (ability to create new chat room, delete accounts, manage groups etc)
Payment will be $ 300 per month ongoing based on performance.
Room specific groups and site wide groups.
Ability for users to create and manage new groups.
Main html/css layout
Future requirements:
Ability to integrate with Twitter accounts and hashtags
Administration panel (ability to create new chat room, delete accounts, manage groups etc)
Payment will be $ 300 per month ongoing based on performance.
Real Estate Website Design
ne. I'm looking to have this up and running ASAP, so we'll talk deadlines and the like after you've taken a look at what I have and what I'm looking for.
NOTE: I had to repost this because the last employee did not communicate with me AT ALL. If you feel like you are too busy to handle a project like this or you are unable/unwilling to communicate regularly, DO NOT BID ON THIS! Thank you :)
NOTE: I had to repost this because the last employee did not communicate with me AT ALL. If you feel like you are too busy to handle a project like this or you are unable/unwilling to communicate regularly, DO NOT BID ON THIS! Thank you :)
Real Estate Website
real estate website using squarespaces.com
