Robin Ward
|
a57f802048
|
If there's a `TopicEmbed` record for a url, we don't have to crawl it.
This should help sites like Boing Boing where sometimes links are
crawled before saved in WordPress.
|
2014-04-17 14:00:22 -04:00 |
|
Robin Ward
|
e80851b0fa
|
Special case: When crawling a link to an image, just put the filename as
the title.
|
2014-04-10 13:45:13 -04:00 |
|
Robin Ward
|
99e2bab62d
|
Use `update_all` to prevent `after_commit` from executing again.
|
2014-04-10 13:19:57 -04:00 |
|
Robin Ward
|
aa63868d5e
|
FIX: Problem crawling amazon titles
|
2014-04-08 16:39:47 -04:00 |
|
Robin Ward
|
1e3faddfe4
|
FIX: Change crawl size to 10k. Youtube for example doesn't work with the
first 1k
|
2014-04-07 16:03:47 -04:00 |
|
Robin Ward
|
7e0028ba50
|
FIX: Don't crawl in test mode, raise correct exception when parameters
are missing
|
2014-04-07 14:38:18 -04:00 |
|
Robin Ward
|
7e3ea5d644
|
Support for crawling topic links
|
2014-04-07 14:08:34 -04:00 |