#include <webcrawler.h>

+ Inheritance diagram for WebCrawler_Spider:
+ Collaboration diagram for WebCrawler_Spider:

Public Slots

void get ()
 Spider main functionality Takes urls from frontier queue and downloads their HTML source code. When http signals finished(), the httpFinished() slot passes the response data to wc_parser thread parse() method to parse them. More...
 
void httpFinished (QNetworkReply *reply)
 Called when QNetworkAccessManager http emits finished() Emits parse with the reply object to WebCrawler_Parser::parse() More...
 

Signals

void parse (QNetworkReply *reply)
 
void finished (QString)
 

Public Member Functions

 WebCrawler_Spider ()
 spider's constructor - does nothing More...
 
 ~WebCrawler_Spider ()
 
void load (const QString &seed, const int &maxNodes, const bool &delayedRequests)
 Called from Graph to init variables. Connects http NetworkManager signal to httpfinished() slot which in turn emits http reply to WebCrawler_Parser. More...
 

Private Attributes

QNetworkAccessManager * http
 
QNetworkRequest request
 
QNetworkReply * reply
 
QUrl currentUrl
 
QString m_seed
 
int m_maxNodes
 
int m_visitedNodes
 
int m_wait_msecs
 
bool m_delayedRequests
 

Constructor & Destructor Documentation

WebCrawler_Spider::WebCrawler_Spider ( )

spider's constructor - does nothing

WebCrawler_Spider::~WebCrawler_Spider ( )

Member Function Documentation

void WebCrawler_Spider::finished ( QString  )
signal
void WebCrawler_Spider::get ( )
slot

Spider main functionality Takes urls from frontier queue and downloads their HTML source code. When http signals finished(), the httpFinished() slot passes the response data to wc_parser thread parse() method to parse them.

void WebCrawler_Spider::httpFinished ( QNetworkReply *  reply)
slot

Called when QNetworkAccessManager http emits finished() Emits parse with the reply object to WebCrawler_Parser::parse()

Parameters
reply
void WebCrawler_Spider::load ( const QString &  url,
const int &  maxN,
const bool &  delayedRequests 
)

Called from Graph to init variables. Connects http NetworkManager signal to httpfinished() slot which in turn emits http reply to WebCrawler_Parser.

Parameters
url
maxN
maxLinksPerPage
extLinks
intLinks
void WebCrawler_Spider::parse ( QNetworkReply *  reply)
signal

Member Data Documentation

QUrl WebCrawler_Spider::currentUrl
private
QNetworkAccessManager* WebCrawler_Spider::http
private
bool WebCrawler_Spider::m_delayedRequests
private
int WebCrawler_Spider::m_maxNodes
private
QString WebCrawler_Spider::m_seed
private
int WebCrawler_Spider::m_visitedNodes
private
int WebCrawler_Spider::m_wait_msecs
private
QNetworkReply* WebCrawler_Spider::reply
private
QNetworkRequest WebCrawler_Spider::request
private

The documentation for this class was generated from the following files: