Skip to content

Conversation

@tinchodev
Copy link

Fixes multibyte character issue on select/multiselect product attributes

@avstudnitz
Copy link
Owner

We have 18 occurences of strtolower. Should they all be replaced by mb_strtolower?

@tinchodev
Copy link
Author

It is always nice to support multibyte characters either for compare or display .
For this particular case, option labels like "NIÑO" were lower-cased to "niÑo" since strtolower cannot handle multibyte characters, hence the validation failed. Nevertheless the change will add mbstring PHP extension as requirement to use the module, which usually is enabled, but it is not a default PHP extension, that will be the only catch.

@avstudnitz
Copy link
Owner

@tinchodev Thanks for clarification. IMO add the mbstring extension as requirement is no big problem. Am I right that we should replace all occurences of strtolower with mb_strtolower then, or are there any additional drawbacks?

@tinchodev
Copy link
Author

Both functions work the same way, technically there are no additional drawbacks.

@avstudnitz
Copy link
Owner

Fixed that in 0a50cc4 for all occurences of strtolower

@avstudnitz avstudnitz closed this Feb 2, 2017
@StevenSch
Copy link

With this this fix im getting the followig error :
Notice: Undefined index: gelöschte artikel in C:\xampp\htdocs\devserver\app\code\community\AvS\FastSimpleImport\Model\Import\Entity\Product\Type\Simple.php on line 33

@avstudnitz
Copy link
Owner

@StevenSch Can you show me the import data you are using where "gelöschte Artikel" occurs?

@avstudnitz avstudnitz reopened this Feb 8, 2017
@StevenSch
Copy link

We planning up a complex multishop-system with more than one root-category (one for each shop).
Dataexchange should be realised with fastsimpleimport and ho_import.
This is a very short snippet from our csv-import-file.
shortlist.txt

@tinchodev
Copy link
Author

@StevenSch "gelöschte Artikel" is not being lowercased for some reason on your setup, I could not reproduce locally but I'm not running Windows which I suspect may be somehow related to the described issue.
Please run a standalone test and share the output :

echo mb_strtolower("gelöschte Artikel", mb_detect_encoding("gelöschte Artikel"));

then

echo strtolower("gelöschte Artikel");

@StevenSch
Copy link

local environment (windows)
php test.php results:
gel├Âschte artikel
gelÒÂschte artikel

This seems to be a windows problem, because on our dev-server (Linux):
php_cli test.php results:
gelöschte artikel
gelöschte artikel

hnsr pushed a commit to ho-nl-fork/AvS_FastSimpleImport that referenced this pull request May 8, 2017
* develop:
  Fixed last missing strtolower replacement by mb_strtolower wrapper
  Add multibyte aware strtolower to attribute-validation
  Bump version to 0.8.0
  Bump version to 0.8.0
  Revert changes in test.php
  Merge branch 'issue154' of git://github.com/barbazul/AvS_FastSimpleImport into barbazul-issue154
  Replace all strtolower occurences by helper method which takes care of multibyte variant (fixes avstudnitz#271)
  Remove unnecessary logs.
  Importing attribute options price (_super_attribute_price_corr) for multiple stores/websites. Defined additional storeCodeToId array, it will take the id from our store code in order to get the website id. We do not have this information in dataRow.
  Adds the missing shell command to modman
  [SKU-Type] SKU-Type Attribute is now inserted
  [Grouped] Fix for undefined variable Warning
  Use pages in _loadSkuSuperAttributeValues() to reduce memory usage
  Override _loadSkuSuperAttributeValues()
  Properly set rowNum during saveProducts
  avstudnitz#327 Load Backend Config when Importing via Backend [possible hack]
  Fix multiple stores
  [FIX] copy magento way to delete old scope specific attribute value
  [FIX] keep category position if set
  [FIX] delete old product attribute if import is not append mode
  Fix category get's emptied when there are more rows in the csv file than category associations.
  Fixes avstudnitz#311
  Fixed Issue That Attribute Options Were Created Twice
  Don't throw exception when external image cannot be downloaded
  Use suggest rather than require
  [fix] add support for PHP 5.3
  Add getNewCustomers, getOldCustomers, getNewCategory and getOldCategory getters
  use multibyte string functions if available
  case-insensitive comparison of dropdown/mutliselect attribute options
  [FIX] delete not all category assignments during catalog_category_product replace
  New version 0.7.0
  Add support for _use_config columns for available_sort_by, default_sort_by and filter_price_range Fixes avstudnitz#154
  Save _bundle_option_title translations.
  Pull request avstudnitz#262
  Pull request avstudnitz#228
  Pull request avstudnitz#227
  Fix getAffectedEntityIds() for category (missing root)
  Fixes possible security issues with excel formula injection in csv data, see APPSEC-1110 (SUPEE-7405).
  Fix getAffectedEntityIds() for category (missing root)
  fixes avstudnitz#154 - Don't validate "required" catelog attributes
@sprankhub
Copy link
Collaborator

The strtolower vs. mb_strtolower issue should be fixed everywhere now.

Windows is not supported by Magento, so we do not support it either.

@sprankhub sprankhub closed this Jul 14, 2017
hnsr pushed a commit to ho-nl-fork/AvS_FastSimpleImport that referenced this pull request Jul 26, 2017
…into only-exisiting-entities

* 'develop' of github.com:avstudnitz/AvS_FastSimpleImport:
  create downloadable/files/links directory if it does not exist yet
  Fixed multi-store downloadable product links import, when "downloadable_links_file" is empty
  Fixed last missing strtolower replacement by mb_strtolower wrapper
  Add multibyte aware strtolower to attribute-validation
  Remove debugging code
  Bump version to 0.8.2
  Only delete category product associtions during product import if at least one category is assigned in 'replace' mode
  Add possibility to import sku_type of bundle products (solves avstudnitz#355)
  Revert 953a998 as it broke the integration tests
  Revert 9ffbeb1 as it made the sku_type field a required value
  Bump version to 0.8.0
  Bump version to 0.8.0
  Revert changes in test.php
  Merge branch 'issue154' of git://github.com/barbazul/AvS_FastSimpleImport into barbazul-issue154
  Replace all strtolower occurences by helper method which takes care of multibyte variant (fixes avstudnitz#271)
  Remove unnecessary logs.
  Importing attribute options price (_super_attribute_price_corr) for multiple stores/websites. Defined additional storeCodeToId array, it will take the id from our store code in order to get the website id. We do not have this information in dataRow.
  Update Import.php
  Adds the missing shell command to modman
  [Downloadable] Links Purchase Attribute is now inserted
  [SKU-Type] SKU-Type Attribute is now inserted
  Added ability to import downloadable products with downloadable links
  [Grouped] Fix for undefined variable Warning
  Use pages in _loadSkuSuperAttributeValues() to reduce memory usage
  Override _loadSkuSuperAttributeValues()
  Properly set rowNum during saveProducts
  avstudnitz#327 Load Backend Config when Importing via Backend [possible hack]
  Fix multiple stores
  [FIX] copy magento way to delete old scope specific attribute value
  [FIX] keep category position if set
  [FIX] delete old product attribute if import is not append mode
  Fix category get's emptied when there are more rows in the csv file than category associations.
  Fixes avstudnitz#311
  Fixed Issue That Attribute Options Were Created Twice
  Don't throw exception when external image cannot be downloaded
  Use suggest rather than require
  [fix] add support for PHP 5.3
  Add getNewCustomers, getOldCustomers, getNewCategory and getOldCategory getters
  use multibyte string functions if available
  case-insensitive comparison of dropdown/mutliselect attribute options
  [FIX] delete not all category assignments during catalog_category_product replace
  New version 0.7.0
  Add support for _use_config columns for available_sort_by, default_sort_by and filter_price_range Fixes avstudnitz#154
  Save _bundle_option_title translations.
  Pull request avstudnitz#262
  Pull request avstudnitz#228
  Pull request avstudnitz#227
  Fix getAffectedEntityIds() for category (missing root)
  Fixes possible security issues with excel formula injection in csv data, see APPSEC-1110 (SUPEE-7405).
  Fix getAffectedEntityIds() for category (missing root)
  fixes avstudnitz#154 - Don't validate "required" catelog attributes
hnsr pushed a commit to ho-nl-fork/AvS_FastSimpleImport that referenced this pull request Jul 26, 2017
…into qj

* 'develop' of github.com:avstudnitz/AvS_FastSimpleImport:
  create downloadable/files/links directory if it does not exist yet
  Fixed multi-store downloadable product links import, when "downloadable_links_file" is empty
  Fixed last missing strtolower replacement by mb_strtolower wrapper
  Add multibyte aware strtolower to attribute-validation
  Remove debugging code
  Bump version to 0.8.2
  Only delete category product associtions during product import if at least one category is assigned in 'replace' mode
  Add possibility to import sku_type of bundle products (solves avstudnitz#355)
  Revert 953a998 as it broke the integration tests
  Revert 9ffbeb1 as it made the sku_type field a required value
  Bump version to 0.8.0
  Bump version to 0.8.0
  Revert changes in test.php
  Merge branch 'issue154' of git://github.com/barbazul/AvS_FastSimpleImport into barbazul-issue154
  Replace all strtolower occurences by helper method which takes care of multibyte variant (fixes avstudnitz#271)
  Remove unnecessary logs.
  Importing attribute options price (_super_attribute_price_corr) for multiple stores/websites. Defined additional storeCodeToId array, it will take the id from our store code in order to get the website id. We do not have this information in dataRow.
  Update Import.php
  Adds the missing shell command to modman
  [Downloadable] Links Purchase Attribute is now inserted
  [SKU-Type] SKU-Type Attribute is now inserted
  Added ability to import downloadable products with downloadable links
  [Grouped] Fix for undefined variable Warning
  Use pages in _loadSkuSuperAttributeValues() to reduce memory usage
  Override _loadSkuSuperAttributeValues()
  Properly set rowNum during saveProducts
  avstudnitz#327 Load Backend Config when Importing via Backend [possible hack]
  Fix multiple stores
  [FIX] copy magento way to delete old scope specific attribute value
  [FIX] keep category position if set
  [FIX] delete old product attribute if import is not append mode
  Fix category get's emptied when there are more rows in the csv file than category associations.
  Fixes avstudnitz#311
  Fixed Issue That Attribute Options Were Created Twice
  Don't throw exception when external image cannot be downloaded
  Use suggest rather than require
  [fix] add support for PHP 5.3
  Add getNewCustomers, getOldCustomers, getNewCategory and getOldCategory getters
  use multibyte string functions if available
  case-insensitive comparison of dropdown/mutliselect attribute options
  [FIX] delete not all category assignments during catalog_category_product replace
  New version 0.7.0
  Add support for _use_config columns for available_sort_by, default_sort_by and filter_price_range Fixes avstudnitz#154
  Save _bundle_option_title translations.
  Pull request avstudnitz#262
  Pull request avstudnitz#228
  Pull request avstudnitz#227
  Fix getAffectedEntityIds() for category (missing root)
  Fixes possible security issues with excel formula injection in csv data, see APPSEC-1110 (SUPEE-7405).
  Fix getAffectedEntityIds() for category (missing root)
  fixes avstudnitz#154 - Don't validate "required" catelog attributes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants