ActiveRecord::Base#find_each is an awesome idiom for batch processing large sets of data. It combines the neat DSL of ActiveRecord and allows you to operate at the level of the individual item.
So, imagine my surprise when I debugged my code and discovered that find_each introduces a scope into ActiveRecord calls within the block. Demonstrating by example:
Slam.find_each(:conditions => {:value => nil}) do |slam|
#….
Slam.find_by_key(slam.key) #Find related key
#However, in effect becomes Slam.find_by_key(:key => slam.key, :value => nil)
end
Looking at the SQL in the logs, I expected to see:
select * from slams where key = ‘abcde’;
Instead I saw:
select * from slams where key = ‘abcde’ and value is null;
It took me fifteen minutes to check and verify there is no code that should be generating that SQL, so the next candidate was find_each. Lo and behold, it was. Turns out find_each is using with_scope.
Looking around, there are already some bug reports for this behaviour:
- https://rails.lighthouseapp.com/projects/8994/tickets/2791-activerecordbasefind_in_batches-puts-a-with_scope-into-the-block-that-is-executed
- https://rails.lighthouseapp.com/projects/8994/tickets/2227-batches-conditions-for-each-are-applied-to-each-modelfind-within-the-each-loop
Update: I have reviewed one promising patch to fix this. Needs one more person to review and approve this patch!