de.aitools.aq.webdownload
Class WgetDownloader

java.lang.Object
  de.aitools.aq.webdownload.Downloader
      de.aitools.aq.webdownload.WgetDownloader

public class WgetDownloader
extends Downloader

Downloader implementation that wraps the wget command. CAUTION: using multiple downloaders in parallel threads eventually causes IOExceptions (too many open connections) while trying to create the process to wrap wget. This occurs because the garbage collection is not fast enough to recycle all process objects. A workaround is to call the garbage collector manually after every n downloads. (A test using 10 parallel downloaders and calling gc every 1000 downloads worked fine.)

Author:

christof.braeutigam@uni-weimar.de

Constructor Summary

WgetDownloader()

WgetDownloader(java.lang.String[] params)

Method Summary

java.io.File	load(java.lang.String url) Loads the data designated by url and creates and returns a file object which can be used to access the data.
java.io.File	load(java.lang.String url, java.io.File dir) Loads the data designated by url to the directory designated by dir.
java.io.File	load(java.net.URL url) Loads the data designated by url and creates and returns a file object which can be used to access the data.
java.io.File	load(java.net.URL url, java.io.File dir) Loads the data designated by url to the directory designated by dir.

Methods inherited from class de.aitools.aq.webdownload.Downloader

getContentType, getContentType, getHeaders, getReadTimeout, getTempFilePrefix, getTempFileSuffix, setConnectionTimeout, setConnectionTimeout, setReadTimeout, setTempFilePrefix, setTempFileSuffix

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WgetDownloader

public WgetDownloader()

WgetDownloader

public WgetDownloader(java.lang.String[] params)

Method Detail

load

public java.io.File load(java.net.URL url)
                  throws DownloaderException

Loads the data designated by url and creates and returns a file object which can be used to access the data. The returned file object represents a temporary file located in the default temp directory.

CAUTION: The file designated by the returned file object will be deleted automatically at a normal JVM exit.

Specified by:

load in class Downloader

Parameters:

url - The resource to download

Returns:

A File to access the downloaded data

Throws:

DownloaderException

load

public java.io.File load(java.lang.String url)
                  throws DownloaderException

Loads the data designated by url and creates and returns a file object which can be used to access the data. The returned file object represents a temporary file located in the default temp directory.

CAUTION: The file designated by the returned file object will be deleted automatically at a normal JVM exit.

Specified by:

load in class Downloader

Parameters:

url - The resource to download

Returns:

A File to access the downloaded data

Throws:

DownloaderException

load

public java.io.File load(java.net.URL url,
                         java.io.File dir)
                  throws DownloaderException

Loads the data designated by url to the directory designated by dir. A file is created in dir and will be returned to provide access to the downloaded data.

Note that the provided dir must denote a directory and this directory must be writable.

CAUTION: The returned file object represents a temporary file located in the directory designated by dir. This file will NOT be deleted automatically, so you have to care for your disk space yourself.

Specified by:

load in class Downloader

Parameters:

url - The resource to download

dir - The directory where the downloaded data should be stored

Returns:

A File to access the downloaded data

Throws:

DownloaderException

load

public java.io.File load(java.lang.String url,
                         java.io.File dir)
                  throws DownloaderException

Loads the data designated by url to the directory designated by dir. A file is created in dir and will be returned to provide access to the downloaded data.

Note that the provided dir must denote a directory and this directory must be writable.

CAUTION: The returned file object represents a temporary file located in the directory designated by dir. This file will NOT be deleted automatically, so you have to care for your disk space yourself.

Specified by:

load in class Downloader

Parameters:

url - The resource to download

dir - The directory where the downloaded data should be stored

Returns:

A File to access the downloaded data

Throws:

DownloaderException