Page (edu.uci.ics:crawler4j 4.4.0 API)

java.lang.Object
- edu.uci.ics.crawler4j.crawler.Page

```
public class Page
extends Object
```
This class contains the data for a fetched and parsed page.

Author:

Yasser Ganjisaffar

Field Summary

Fields
Modifier and Type	Field and Description
`protected String`	`contentCharset` The charset of the content.
`protected byte[]`	`contentData` The content of this page in binary format.
`protected String`	`contentEncoding` The encoding of the content.
`protected String`	`contentType` The ContentType of this page.
`protected org.apache.http.Header[]`	`fetchResponseHeaders` Headers which were present in the response of the fetch request
`protected org.slf4j.Logger`	`logger`
`protected ParseData`	`parseData` The parsed data populated by parsers
`protected boolean`	`redirect` Redirection flag
`protected String`	`redirectedToUrl` The URL to which this page will be redirected to
`protected int`	`statusCode` Status of the page
`protected boolean`	`truncated` Whether the content was truncated because the received data exceeded the imposed maximum
`protected WebURL`	`url` The URL of this page.

Constructor Summary

Constructors
Constructor and Description

Page(WebURL url)

Constructors
Constructor and Description
`Page(WebURL url)`

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`String`	`getContentCharset()`
`byte[]`	`getContentData()`
`String`	`getContentEncoding()`
`String`	`getContentType()`
`org.apache.http.Header[]`	`getFetchResponseHeaders()` Returns headers which were present in the response of the fetch request
`String`	`getLanguage()`
`ParseData`	`getParseData()`
`String`	`getRedirectedToUrl()`
`int`	`getStatusCode()`
`WebURL`	`getWebURL()`
`boolean`	`isRedirect()`
`boolean`	`isTruncated()`
`void`	`load(org.apache.http.HttpEntity entity, int maxBytes)` Loads the content of this page from a fetched HttpEntity.
`void`	`setContentCharset(String contentCharset)`
`void`	`setContentData(byte[] contentData)`
`void`	`setContentEncoding(String contentEncoding)`
`void`	`setContentType(String contentType)`
`void`	`setFetchResponseHeaders(org.apache.http.Header[] headers)`
`void`	`setLanguage(String language)`
`void`	`setParseData(ParseData parseData)`
`void`	`setRedirect(boolean redirect)`
`void`	`setRedirectedToUrl(String redirectedToUrl)`
`void`	`setStatusCode(int statusCode)`
`void`	`setWebURL(WebURL url)`
`protected byte[]`	`toByteArray(org.apache.http.HttpEntity entity, int maxBytes)` Read contents from an entity, with a specified maximum.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - logger
```
protected final org.slf4j.Logger logger
```
  - url
```
protected WebURL url
```
    The URL of this page.
  - redirect
```
protected boolean redirect
```
    Redirection flag
  - redirectedToUrl
```
protected String redirectedToUrl
```
    The URL to which this page will be redirected to
  - statusCode
```
protected int statusCode
```
    Status of the page
  - contentData
```
protected byte[] contentData
```
    The content of this page in binary format.
  - contentType
```
protected String contentType
```
    The ContentType of this page. For example: "text/html; charset=UTF-8"
  - contentEncoding
```
protected String contentEncoding
```
    The encoding of the content. For example: "gzip"
  - contentCharset
```
protected String contentCharset
```
    The charset of the content. For example: "UTF-8"
  - fetchResponseHeaders
```
protected org.apache.http.Header[] fetchResponseHeaders
```
    Headers which were present in the response of the fetch request
  - parseData
```
protected ParseData parseData
```
    The parsed data populated by parsers
  - truncated
```
protected boolean truncated
```
    Whether the content was truncated because the received data exceeded the imposed maximum
- Constructor Detail
  - Page
```
public Page(WebURL url)
```
- Method Detail
  - toByteArray
```
protected byte[] toByteArray(org.apache.http.HttpEntity entity,
                             int maxBytes)
                      throws IOException
```
    Read contents from an entity, with a specified maximum. This is a replacement of EntityUtils.toByteArray because that function does not impose a maximum size.
    
    Parameters:
    
    entity - The entity from which to read
    
    maxBytes - The maximum number of bytes to read
    
    Returns:
    
    A byte array containing maxBytes or fewer bytes read from the entity
    
    Throws:
    
    IOException - Thrown when reading fails for any reason
  - load
```
public void load(org.apache.http.HttpEntity entity,
                 int maxBytes)
          throws Exception
```
    Loads the content of this page from a fetched HttpEntity.
    
    Parameters:
    
    entity - HttpEntity
    
    maxBytes - The maximum number of bytes to read
    
    Throws:
    
    Exception - when load fails
  - getWebURL
```
public WebURL getWebURL()
```
  - setWebURL
```
public void setWebURL(WebURL url)
```
  - isRedirect
```
public boolean isRedirect()
```
  - setRedirect
```
public void setRedirect(boolean redirect)
```
  - getRedirectedToUrl
```
public String getRedirectedToUrl()
```
  - setRedirectedToUrl
```
public void setRedirectedToUrl(String redirectedToUrl)
```
  - getStatusCode
```
public int getStatusCode()
```
  - setStatusCode
```
public void setStatusCode(int statusCode)
```
  - getFetchResponseHeaders
```
public org.apache.http.Header[] getFetchResponseHeaders()
```
    Returns headers which were present in the response of the fetch request
    
    Returns:
    
    Header Array, the response headers
  - setFetchResponseHeaders
```
public void setFetchResponseHeaders(org.apache.http.Header[] headers)
```
  - getParseData
```
public ParseData getParseData()
```
    Returns:
    
    parsed data generated for this page by parsers
  - setParseData
```
public void setParseData(ParseData parseData)
```
  - getContentData
```
public byte[] getContentData()
```
    Returns:
    
    content of this page in binary format.
  - setContentData
```
public void setContentData(byte[] contentData)
```
  - getContentType
```
public String getContentType()
```
    Returns:
    
    ContentType of this page. For example: "text/html; charset=UTF-8"
  - setContentType
```
public void setContentType(String contentType)
```
  - getContentEncoding
```
public String getContentEncoding()
```
    Returns:
    
    encoding of the content. For example: "gzip"
  - setContentEncoding
```
public void setContentEncoding(String contentEncoding)
```
  - getContentCharset
```
public String getContentCharset()
```
    Returns:
    
    charset of the content. For example: "UTF-8"
  - setContentCharset
```
public void setContentCharset(String contentCharset)
```
  - getLanguage
```
public String getLanguage()
```
    Returns:
    
    Language
  - setLanguage
```
public void setLanguage(String language)
```
  - isTruncated
```
public boolean isTruncated()
```

Class Page

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

logger

url

redirect

redirectedToUrl

statusCode

contentData

contentType

contentEncoding

contentCharset

fetchResponseHeaders

parseData

truncated

Constructor Detail

Page

Method Detail

toByteArray

load

getWebURL

setWebURL

isRedirect

setRedirect

getRedirectedToUrl

setRedirectedToUrl

getStatusCode

setStatusCode

getFetchResponseHeaders

setFetchResponseHeaders

getParseData

setParseData

getContentData

setContentData

getContentType

setContentType

getContentEncoding

setContentEncoding

getContentCharset

setContentCharset

getLanguage

setLanguage

isTruncated