deac/activerecord-import
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity
103 commits 1 branch 16 tags 3.7 MiB
Commit graph

49 commits

Author SHA1 Message Date
Zach Dennis b2c71336ca Merge pull request #20 from Empact/drop_sum_sizes 
Rely on ActiveSupport #sum rather than #sum_sizes
 2012-01-06 15:47:53 -08:00
Yehuda Katz b5261d0225 Clean up module usage. There's no need to use InstanceMethods -- normal Ruby modules and includes will work fine here. 2012-01-03 00:41:04 -08:00
Nat Budin 08c36d33e4 Use connection pool instead of actual connection to avoid having to connect to the database until necessary 2011-10-10 13:07:10 -04:00
Ben Woosley 8d20594f12 Replace #sum_sizes with calls to ActiveSupport #sum, as splatting large numbers of values is resource intensive andcan cause stack overflows of the sort 'SystemStackError: stack level too deep' 2011-08-16 17:01:11 -07:00
Zach Dennis 5a71efa2b4 Merge pull request #21 from jamiecobbett/master 
Fix import of very large datasets containing multibyte chars on Ruby 1.9
 2011-06-22 19:51:29 -07:00
Zach Dennis 92ec1b478d Merge pull request #26 from fallwith/master 
Ruby 1.9.2 / RubyGems 1.8.5 compatibility fix (pass String instead of Pathname to require)
 2011-06-22 19:45:17 -07:00
fallwith 8f91eeb93d Regexp.escape which is called by Gem.loaded_path? (/usr/local/lib/ruby/site_ruby/1.9.1/rubygems.rb) expects the path as a String, not a Pathname. Convert Pathnames to Strings prior to calling 'require' 2011-06-22 16:01:33 -07:00
Jamie Cobbett 6555fd70ba Correctly obey MySQL MAX PACKET 
Use String's bytesize instead of size, because we're talking about byte length,
not string length. This resolves a problem on Ruby 1.9 with multi-byte
characters.
 2011-05-08 21:35:47 +00:00
Jason 88e2ee034b Adding jdbcmysql support. 2011-05-04 20:27:11 -04:00
Zach Dennis 5aefbb2989 Do not use ActiveRecord::Base.new(hsh) when instantiating AR models. Use form of instantiation that allows protected attributes to be set. 2011-04-29 00:13:26 -04:00
Zach Dennis 89662a67c2 Updating documentation for synchronize. 
https://github.com/zdennis/activerecord-import/issues/16
 2011-04-06 14:41:42 -04:00
Zach Dennis 58b144b6bf Updating documentation for synchronizing unsaved/new instances with import. 
https://github.com/zdennis/activerecord-import/issues/16
 2011-04-06 14:23:13 -04:00
Zach Dennis 7958ed18c2 Added :synchronize_keys to the import options so importing can synchronize on fields beside the primary key. 
https://github.com/zdennis/activerecord-import/issues/16
 2011-04-06 14:20:53 -04:00
Zach Dennis 319c52d80f Pulled in ActiveRecord synchronize functionality from the old ar-extensions code base. 
https://github.com/zdennis/activerecord-import/issues#issue/6
 2011-03-19 15:38:24 -04:00
Franck Verrot d901a7c60f Importing an empty array of objects/values should be supported 2011-03-19 15:11:28 -04:00
Zach Dennis cb6b726eac Ensure the #max_allowed_packet method exists for the MysqlAdapters. 
https://github.com/zdennis/activerecord-import/issues#issue/12
 2011-01-11 21:01:34 -05:00
Zach Dennis 161cf017f9 Renamed #values_sql_for_column_names_and_attributes to #values_sql_for_columns_and_attributes and cleaned up how it dealt with columns. 
* added back in the support for type-casting based on the database column
* removed unused method in ActiveRecord::Import::AbstractAdapter

https://github.com/zdennis/activerecord-import/issues#issue/5
 2011-01-10 22:36:52 -05:00
Ben Woosley 1a8082444b Simplify validations_arry_for_column_names_and_attributes with map, each_with_index and Hash. 
Hash[column_names.zip(attributes)] would be much clearer, but would leave nil values for missing attributes which doesn't match the existing behavior.
 2011-01-10 22:27:24 -05:00
Ben Woosley c9f246f4e2 Consolidate insert_sql between supports_import? cases so both make use of options[:ignore] 2011-01-10 22:27:23 -05:00
Ben Woosley 9727a04362 Reinstate the deleting of the :validate option - not because it was causing trouble, but to be a bit paranoid (reverts 63d6faeccee14b6e571a25b6f51f340d5da325f5) 2011-01-10 22:27:23 -05:00
Ben Woosley a3e9b4f040 Enable ImportSupport on Postgresql - the test suite now passes in full 2011-01-10 22:27:23 -05:00
Ben Woosley 7bd0e2693a Fix that values_sql_for_attributes wasn't accessing the columns with the same indexes as the array_of_attributes, and so was sometime mismatched. 
Also return to the column_names[j] == primary_key style of testing, as the column itself is not always present
 2011-01-10 22:27:22 -05:00
Ben Woosley df75eea4db Go more direct in the columns lookup 2011-01-10 22:26:39 -05:00
Ben Woosley 35786fef17 Codify import results in a Struct rather than OpenStruct, as it's a faster, more rigid structure 2011-01-10 22:12:34 -05:00
Ben Woosley 876546c584 #dup array_of_attributes a bit earlier - there's no point in duping the primary_key nulls we might put in next 2011-01-10 22:12:34 -05:00
Ben Woosley 899af1e412 No reason to pull :validate out of the options hash, and it's obfuscatory - remove. 2011-01-10 22:12:34 -05:00
Ben Woosley 18012bbbca More simplicity, once again with #map 2011-01-10 22:12:33 -05:00
Ben Woosley 3756e4f2b4 Refactor remaining supports_import? split to share code in values_sql_for_column_names_and_attributes. This means both with properly support sequences. 
Rather than pass a ton of variables (sequence_name, columns, column_names, &c.), move the method out of the connection.  Also, use Column#primary to check primary key status.
 2011-01-10 22:12:29 -05:00
Ben Woosley 5f4d9b3290 Extract columns_sql local more generally in #import_without_validations_or_callbacks. This is DRY-er, more consistent between the supports_import? cases and enables us to inline #quote_column_names 2011-01-10 22:11:54 -05:00
Ben Woosley 6b2153a841 Simplify values_sql_for_column_names_and_attributes with #map 2011-01-10 22:11:51 -05:00
Ben Woosley 7aefcf1164 Inline multiple_value_sets_insert_sql because it never changes, isn't particularly well-factored, and for parity with "not supports_import?" 2011-01-10 21:57:41 -05:00
Ben Woosley 2ada56f858 Avoid N << method calls in #import_without_validations_or_callbacks by chaining #map (is simpler, should be more performant, AFAIK) 2011-01-10 21:57:40 -05:00
Ben Woosley fff00af942 Don't initialize options[:on_duplicate_key_update] if the adapter doesn't #supports_on_duplicate_key_update? 2011-01-10 21:57:40 -05:00
Zach Dennis 8fbf841fde Use ActiveRecord Column#type_cast to properly parse fields represented by a string. 
Fixed issued reported by Nick Burdick where imported datetime fields was not correctly working with certain string formats for MySQL which doesn't support MM/DD/YYYY format directly.
 2011-01-05 19:09:55 -05:00
Zach Dennis cb542a86f4 Updated how activerecord-import loads so it is done after a connection is established in ActiveRecord. This supports loading import for multiple adapters automatically w/o having to have a person require those specifically. 2010-09-26 20:48:14 -04:00
Zach Dennis 163f5c64be Deprecating usage of "require 'activerecord-import/<adapter_name>'" in favor of "require 'activerecord-import'". 2010-09-26 17:07:41 -04:00
Zach Dennis 90057c43eb Renaming ActiveRecord::Extensions to ActiveRecord::Import. 2010-09-26 17:05:02 -04:00
Zach Dennis 948dec0be2 Refactoring to allow for separation of activerecord-import code and activerecord's connection adapters. 2010-09-26 16:51:19 -04:00
Zach Dennis b6c72f8456 Providing support for the Mysql2 adapter. 2010-09-26 16:04:00 -04:00
Zach Dennis e0dff4c055 Moved all adapter specific code into adapters/ directory and updated all adapters to utilize the same "require_adapter" method of loading specific-database adapters. 
* Adds back in SQLite3 and PostgreSQL support

issue-url: http://github.com/zdennis/activerecord-import/issues#issue/1 
[contributes #1]
 2010-08-10 20:46:32 -04:00
Zach Dennis 4fcfa7fd14 Restructuring directories. 2010-06-26 15:43:34 -04:00
Zach Dennis 5c287f1042 Added back in basic benchmark support. To run: 
ruby benchmark/benchmark.rb --adapter mysql --num 1000
  ruby benchmark/benchmark.rb --adapter mysql --num 1000 --to-csv /tmp/results.csv
  ruby benchmark/benchmark.rb --adapter mysql --num 1000 --to-html /tmp/results.html
 2010-04-08 23:12:28 -04:00
Zach Dennis 3523138a7a * Added basic import support for PostgreSQL. 
* Requiring database adapter is now "require 'ar-extensions/import/<adapter_name>" so it won't conflict with previous versions of ar-extensions or other libraries extracted out of older ar-extensions, which "require 'ar-extensions'"
 2010-04-08 20:57:10 -04:00
Zach Dennis 075104a944 Finished porting over MySQL import functionality. 
* removed assertions on #num_inserts, that is db-specific and is handled in details by abstract_adapter_test.
 2010-03-13 22:30:29 -05:00
Zach Dennis e8271778b7 About 85% through porting over MySQL on duplicate key functionality. 2010-03-13 21:33:03 -05:00
Zach Dennis 5836e449fd Removing unnecessary logger creation. 2010-03-13 20:26:17 -05:00
Zach Dennis d4c120497e Minor refactoring. 2010-03-13 20:20:23 -05:00
Zach Dennis 5c5bd5f716 Implemented tests for #get_value_insert_sets. 2010-03-12 00:04:15 -05:00
Zach Dennis ca5f83e1cf Initial commit of making ar-extensions import Rails 3 friendly 2010-03-11 21:32:56 -05:00