You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I was wonder why the make_request_from_data method doesn't just use the more full featured request_from_dict function from scrapy utils that the queue class in the library already uses? It seems that both do the same thing but even the documentation is a bit misleading when it says:
this data can be accessed in scrapy spider through response. like: request.url, request.meta, request.cookies
as nothing outside of url, method, meta get set as keyword args anyway.
The text was updated successfully, but these errors were encountered:
At the time, request_from_dict required several keys as you can see here. And the need was to load start URLs from redis, which initially were just URL strings, and then changed to dict-like with url, method and meta which were the fields some users needed.
I'm not sure if current request_from_dict can replace current approach seamlessly, feel free to contribute!
IMO, we don't need request_from_dict at the moment. The very first concept of make_request_from_data is making our request more flexible, on the other hand, request_from_dict is letting users define and pass their own request class.
For the above reason, I don't think that we should add request_from_dict now. But discussions are welcome, and feel free to contribute.
I was wonder why the
make_request_from_data
method doesn't just use the more full featuredrequest_from_dict
function from scrapy utils that the queue class in the library already uses? It seems that both do the same thing but even the documentation is a bit misleading when it says:this data can be accessed in scrapy spider through response. like: request.url, request.meta, request.cookies
as nothing outside of
url, method, meta
get set as keyword args anyway.The text was updated successfully, but these errors were encountered: