mass assignment - ActiveRecord#create does not allow to set the id -
i need batch insert > 100.000 records. id not created db , have utilize a given uuid: doing in loop using mymodel.new assigning id, save record work way slow (appr. 20min)
when create array 'records' , utilize mymodel.create(records) run 'cannot mass assign id' problem. i've tried solutions find:
'attr_acccessible :id, ...' model. works id. (re)define 'def self.attributes_protected_by_default [] end' - no effect one advice utilize 'create' ':without_protection => true', create not take more 1 argument..so neither of these solutions helped. else can do?
finally, found solution might not elegant in rails way solves performance problem:
at first tried @albin suggested find create(records) not work much faster (still > 15min).
my solution is:
create temporary csv file
db_tmp = file.open("tmp_file", "w") records = "" @data_records.each |row| records << "#{row['id']},#{row['id']},#{field_1},#{row['field_2']}, ... \n" end db_tmp.write(records) db_tmp.close
execute sql load info command
sql = "load info infile 'tmp_file' table my_table fields optionally enclosed '\"' terminated ',' (id,field_1,field_2, ... )" activerecord::base.connection.execute(sql)
the whole process lasts less 1 (!) minute, including getting info on network , parsing original json message hash.
i'm aware not clarify how create tricked allowing id assignment performance problem solved.
another point solution bypasses validation defined model. not problem because in case know can rely on integrity of info i'm receiving - , if there's problem load fail , execute raise exception.
activerecord mass-assignment
No comments:
Post a Comment