=========
converter
=========

Our ScrapyItemBase class uses ScrapyFieldProperty for define it's fields. Such
fields can use converters whihc are able to convert given input values.
Let's take a look at the different converters

  >>> import s01.scrapy.converter


datetimeConverter
-----------------

Our date time converter tries to convert a given string to a datetime object.
The converter is using a method called getDateFormats which returns common
datetime formatter strings:

  >>> for dtStr in s01.scrapy.converter.getDateFormats():
  ...     print dtStr
  %d.%m.%Y
  %d.%m.%Y %H
  %d.%m.%Y %H:%M
  %d.%m.%Y %H:%M:%S
  %d-%m-%Y
  %d-%m-%Y %H
  %d-%m-%Y %H:%M
  %d-%m-%Y %H:%M:%S
  %Y.%m.%d
  %Y.%m.%d %H
  %Y.%m.%d %H:%M
  %Y.%m.%d %H:%M:%S
  %Y-%m-%d
  %Y-%m-%d %H
  %Y-%m-%d %H:%M
  %Y-%m-%d %H:%M:%S

Let's try to convert a date time string:

  >>> s01.scrapy.converter.datetimeConverter('2010-12-31')
  datetime.datetime(2010, 12, 31, 0, 0)


emailConverter
--------------

The email convert can convert list or simple string to an unicode email adress.
But of corse only if the email adress is valid:

  >>> s01.scrapy.converter.emailConverter(' foo@bar.com ')
  u'foo@bar.com'

  >>> s01.scrapy.converter.emailConverter('foo @ bar.com') is None
  True

  >>> s01.scrapy.converter.emailConverter('.foo@bar.com ') is None
  True

  >>> s01.scrapy.converter.emailConverter('foo@bar.com.') is None
  True

  >>> s01.scrapy.converter.emailConverter('foobar.com') is None
  True

test ()<>@,;:\\"[]

  >>> s01.scrapy.converter.emailConverter('foo\\bar.com') is None
  True

  >>> s01.scrapy.converter.emailConverter('foo\nbar.com') is None
  True

  >>> s01.scrapy.converter.emailConverter('foor\r\nbar.com') is None
  True

  >>> s01.scrapy.converter.emailConverter('foor(@)bar.com') is None
  True

  >>> s01.scrapy.converter.emailConverter('foo;@bar.com') is None
  True

  >>> s01.scrapy.converter.emailConverter('foo:@bar.com') is None
  True

  >>> s01.scrapy.converter.emailConverter('foo"@bar.com') is None
  True

  >>> s01.scrapy.converter.emailConverter('foo[@bar.com') is None
  True

  >>> s01.scrapy.converter.emailConverter('foo]@bar.com') is None
  True

  >>> s01.scrapy.converter.emailConverter('<foo@bar.com>') is None
  True

  >>> s01.scrapy.converter.emailConverter('foo,@bar.com') is None
  True



intConverter
--------------

The int convert can convert list or simple string to an integer:

  >>> s01.scrapy.converter.intConverter(' 42 ')
  42


listConverter
-------------

The list converter can convert an input string to a list of unicode items:

  >>> s01.scrapy.converter.listConverter('foo')
  [u'foo']


textConverter
-------------

The text converter can convert text to unicode:

  >>> print s01.scrapy.converter.textConverter('foo\n\rbar')
  foo
  bar


textLineConverter
-----------------

The text line converter can convert text to unicode and remove linebreaks:

  >>> print s01.scrapy.converter.textLineConverter('foo\n\rbar')
  foo bar


uriConverter
------------

The uri converter can convert given input to a valid uri string. but of corse
only it the givne input is (almost) an uri. Or let's say the convert only
replaces empty spaces and converts unicode input to a string

  >>> s01.scrapy.converter.uriConverter(u'http://foo.bar.com/foo bar/')
  'http://foo.bar.com/foo%20bar/'

  >>> s01.scrapy.converter.uriConverter(u'https://foo.bar.com/foo bar/')
  'https://foo.bar.com/foo%20bar/'

  >>> s01.scrapy.converter.uriConverter(u'ftp://foo.bar.com/foo bar/')
  'ftp://foo.bar.com/foo%20bar/'

  >>> s01.scrapy.converter.uriConverter(u'javascript:window.print()')
  'javascript:window.print()'


httpConverter
-------------

The httpConverter converter can convert given input to a http uri string. If
the given input doesn't start with http or https, the return value will be
None:

  >>> s01.scrapy.converter.httpConverter(u'http://foo.bar.com/foo bar/')
  'http://foo.bar.com/foo%20bar/'

  >>> s01.scrapy.converter.httpConverter(u'https://foo.bar.com/foo bar/')
  'https://foo.bar.com/foo%20bar/'

  >>> s01.scrapy.converter.httpConverter(u'ftp://foo.bar.com/foo bar/') is None
  True

  >>> s01.scrapy.converter.httpConverter(u'javascript:window.print()') is None
  True

