Import CVS "exceeded the maximum allowed size (500 symbols)"

Mario_Canevarollo · April 10, 2020, 1:54pm

Hi! I’m in trouble with a data import.

This is the cvs file:
“ArtNo({”“constraints”":[],"“type”":"“STRING”","“dataSize”":36,"“dataSizeAuto”":true,"“defaultValue”":"“00000000"”})",“description({”“constraints”":[],"“type”":"“TEXT”"})",“expireDate({”“constraints”":["“NN”"],"“type”":"“DATETIME”","“defaultValue”":"“2019-08-09 22:00:00"”})",“img({”“constraints”":["“NN”"],"“type”":"“FILE_REF”","“dataSize”":500,"“defaultValue”":"“https://backendlessappcontent.com/MY-PERSONAL-DATA/files/view/deals/no-image-deal.jpg”"})",“maxQuantity({”“constraints”":["“NN”"],"“type”":"“INT”","“defaultValue”":"“0"”})",“newPrice({”“constraints”":["“NN”"],"“type”":"“DOUBLE”","“defaultValue”":"“0"”})",“note({”“constraints”":[],"“type”":"“STRING”","“dataSize”":44,"“dataSizeAuto”":true,"“defaultValue”":"“Text”"})",“priceCurrency({”“constraints”":["“NN”"],"“type”":"“STRING”","“dataSize”":36,"“dataSizeAuto”":true,"“defaultValue”":"“€”"})",“startDate({”“constraints”":["“NN”"],"“type”":"“DATETIME”","“defaultValue”":"“2019-08-06 22:00:00"”})",“title({”“constraints”":["“NN”"],"“type”":"“STRING”","“dataSize”":78,"“dataSizeAuto”":true,"“defaultValue”":"“Titel des Deal”"})",“volumePrice({”“constraints”":[],"“type”":"“DOUBLE”","“defaultValue”":"“0"”})",“volumeType({”“constraints”":[],"“type”":"“STRING”","“dataSize”":36,"“dataSizeAuto”":true,"“defaultValue”":""""})"
AP20200651,Die Beschreibung folgt in kurzem…,Thu Mar 26 18:04:36 UTC 2020,https://backendlessappcontent.com/MY-PERSONAL-DATA/files/view/deals/fabio-esposito/AP20200651.jpg,100000,5.87,Paluani/Mughett,EUR,Thu Mar 26 18:04:36 UTC 2020,COLOMBA CLASSICA CON CANDITI,11.74,Kg
AP20200602,Die Beschreibung folgt in kurzem…,Thu Mar 26 18:04:36 UTC 2021,https://backendlessappcontent.com/MY-PERSONAL-DATA/files/view/deals/fabio-esposito/AP20200602.jpg,100000,1.94,Rio Mare,EUR,Thu Mar 26 18:04:36 UTC 2021,TONNO IN OLIO RIO MARE 80gr,12.125,Kg
AP20200599,Die Beschreibung folgt in kurzem…,Thu Mar 26 18:04:36 UTC 2022,https://backendlessappcontent.com/MY-PERSONAL-DATA/files/view/deals/fabio-esposito/AP20200599.jpg,100000,0.78,SAI,EUR,Thu Mar 26 18:04:36 UTC 2022,SALE FINO (FEINGEMAHLEN),0.78,Kg
AP20200598,Die Beschreibung folgt in kurzem…,Thu Mar 26 18:04:36 UTC 2023,https://backendlessappcontent.com/MY-PERSONAL-DATA/files/view/deals/fabio-esposito/AP20200598.jpg,100000,0.78,SAI,EUR,Thu Mar 26 18:04:36 UTC 2023,SALE GROSSO (GROB GEMAHLEN),0.78,Kg

This the error from the log file:
13:51:30 IMPORT_DATA Importing Started.
13:51:30 IMPORT_DATA Preparing files to import
13:51:30 IMPORT_DATA Copying of Deals.csv, size: 0.003 MiB.
13:51:30 IMPORT_DATA Creating temporary tables
13:51:30 IMPORT_DATA Creating user’s tables
13:51:30 IMPORT_DATA Preparing Deals table: initializing columns.
13:51:31 IMPORT_DATA Import of tables failed: Property value for “description” exceeded the maximum allowed size (500 symbols)
13:51:31 IMPORT_DATA Importing failed. Exception:com.backendless.exceptions.persistence.InvalidEntitySizeException: Property value for “description” exceeded the maximum allowed size (500 symbols)

	com.backendless.management.impex.manager.imports.data.CommonDatabaseUtils.ensureDataSize(CommonDatabaseUtils.java:48)
	com.backendless.management.impex.manager.imports.data.ImportDataManager.initColumn(ImportDataManager.java:761)
	com.backendless.management.impex.manager.imports.data.ImportDataManager.initColumns(ImportDataManager.java:746)
	com.backendless.management.impex.manager.imports.data.ImportDataManager.initTables(ImportDataManager.java:692)
	com.backendless.tasks.impex.ImportDataTask.doExecute(ImportDataTask.java:173)
	com.backendless.tasks.impex.ImportDataTask.executeImpl(ImportDataTask.java:124)
	com.backendless.tasks.impex.AbstractImportTask.execute(AbstractImportTask.java:83)
	com.backendless.taskman.Task.run(Task.java:136)
	com.backendless.taskman.TimeoutFixedThreadPoolExecutor.executeTaskWithTracker(TimeoutFixedThreadPoolExecutor.java:91)
	com.backendless.taskman.TimeoutFixedThreadPoolExecutor.lambda$invoke$0(TimeoutFixedThreadPoolExecutor.java:39)
	java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	java.base/java.lang.Thread.run(Thread.java:834)

Andriy_Konoz · April 10, 2020, 2:11pm

Hi @Mario_Canevarollo!
Can you please attach content of CSV file as CSV-file?

Regards, Andriy

Mario_Canevarollo · April 10, 2020, 2:29pm

But I wont to share with all of the people here my personal information, you can copy and paste the text above in new cvs file.

Andriy_Konoz · April 10, 2020, 2:32pm

I tried to create CSV file from data from the topic but my PC had detected several kinds of " symbol. By this reason I got invalid file.

Can you send here or in direct messages file that contain only data from this topic?

Regards, Andriy

Mario_Canevarollo · April 10, 2020, 2:38pm

"ArtNo({""constraints"":[],""type"":""STRING"",""dataSize"":36,""dataSizeAuto"":true,""defaultValue"":""00000000""})","description({""constraints"":[],""type"":""TEXT""})","expireDate({""constraints"":[""NN""],""type"":""DATETIME"",""defaultValue"":""2019-08-09 22:00:00""})","img({""constraints"":[""NN""],""type"":""FILE_REF"",""dataSize"":500,""defaultValue"":""https://backendlessappcontent.com/MY-DATA/files/view/deals/no-image-deal.jpg""})","maxQuantity({""constraints"":[""NN""],""type"":""INT"",""defaultValue"":""0""})","newPrice({""constraints"":[""NN""],""type"":""DOUBLE"",""defaultValue"":""0""})","note({""constraints"":[],""type"":""STRING"",""dataSize"":44,""dataSizeAuto"":true,""defaultValue"":""Text""})","priceCurrency({""constraints"":[""NN""],""type"":""STRING"",""dataSize"":36,""dataSizeAuto"":true,""defaultValue"":""€""})","startDate({""constraints"":[""NN""],""type"":""DATETIME"",""defaultValue"":""2019-08-06 22:00:00""})","title({""constraints"":[""NN""],""type"":""STRING"",""dataSize"":78,""dataSizeAuto"":true,""defaultValue"":""Titel des Deal""})","volumePrice({""constraints"":[],""type"":""DOUBLE"",""defaultValue"":""0""})","volumeType({""constraints"":[],""type"":""STRING"",""dataSize"":36,""dataSizeAuto"":true,""defaultValue"":""""})"
AP20200651,Die Beschreibung folgt in kurzem...,Thu Mar 26 18:04:36 UTC 2020,https://backendlessappcontent.com/MY-DATA/files/view/deals/fabio-esposito/AP20200651.jpg,100000,5.87,Paluani/Mughett,EUR,Thu Mar 26 18:04:36 UTC 2020,COLOMBA CLASSICA CON CANDITI,11.74,Kg
AP20200602,Die Beschreibung folgt in kurzem...,Thu Mar 26 18:04:36 UTC 2021,https://backendlessappcontent.com/MY-DATA/files/view/deals/fabio-esposito/AP20200602.jpg,100000,1.94,Rio Mare,EUR,Thu Mar 26 18:04:36 UTC 2021,TONNO IN OLIO RIO MARE 80gr,12.125,Kg
AP20200599,Die Beschreibung folgt in kurzem...,Thu Mar 26 18:04:36 UTC 2022,https://backendlessappcontent.com/MY-DATA/files/view/deals/fabio-esposito/AP20200599.jpg,100000,0.78,SAI,EUR,Thu Mar 26 18:04:36 UTC 2022,SALE FINO  (FEINGEMAHLEN),0.78,Kg
AP20200598,Die Beschreibung folgt in kurzem...,Thu Mar 26 18:04:36 UTC 2023,https://backendlessappcontent.com/MY-DATA/files/view/deals/fabio-esposito/AP20200598.jpg,100000,0.78,SAI,EUR,Thu Mar 26 18:04:36 UTC 2023,SALE GROSSO (GROB GEMAHLEN),0.78,Kg

Andriy_Konoz · April 10, 2020, 3:09pm

I tried to import your data to test app - there was no problems with the size of values but there was problem with datetime values. You must provide timestamp with milliseconds for datetime columns. Under my message you will find attached file with valid data (I chaged your datetime values for random timestamp in it).

As for original problem - it is caused by data that you did not sent to me. I think, that you have values larger than 500 characters in “img” column. Try to encrease colum size. Backendless max allowed size for columns of such type is 2000 characters.

Regards, Andriytest.csv (2.1 KB)

Mario_Canevarollo · April 10, 2020, 4:58pm

So, I tried to import your new file (I renamed it with the table’s name) but I got the same error:
16:56:00 IMPORT_DATA Importing Started.
16:56:00 IMPORT_DATA Preparing files to import
16:56:00 IMPORT_DATA Copying of Deals.csv, size: 0.002 MiB.
16:56:00 IMPORT_DATA Creating temporary tables
16:56:00 IMPORT_DATA Creating user’s tables
16:56:00 IMPORT_DATA Preparing Deals table: initializing columns.
16:56:01 IMPORT_DATA Import of tables failed: Property value for “description” exceeded the maximum allowed size (500 symbols)
16:56:01 IMPORT_DATA Importing failed. Exception:com.backendless.exceptions.persistence.InvalidEntitySizeException: Property value for “description” exceeded the maximum allowed size (500 symbols)

	com.backendless.management.impex.manager.imports.data.CommonDatabaseUtils.ensureDataSize(CommonDatabaseUtils.java:48)
	com.backendless.management.impex.manager.imports.data.ImportDataManager.initColumn(ImportDataManager.java:761)
	com.backendless.management.impex.manager.imports.data.ImportDataManager.initColumns(ImportDataManager.java:746)
	com.backendless.management.impex.manager.imports.data.ImportDataManager.initTables(ImportDataManager.java:692)
	com.backendless.tasks.impex.ImportDataTask.doExecute(ImportDataTask.java:173)
	com.backendless.tasks.impex.ImportDataTask.executeImpl(ImportDataTask.java:124)
	com.backendless.tasks.impex.AbstractImportTask.execute(AbstractImportTask.java:83)
	com.backendless.taskman.Task.run(Task.java:136)
	com.backendless.taskman.TimeoutFixedThreadPoolExecutor.executeTaskWithTracker(TimeoutFixedThreadPoolExecutor.java:91)
	com.backendless.taskman.TimeoutFixedThreadPoolExecutor.lambda$invoke$0(TimeoutFixedThreadPoolExecutor.java:39)
	java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	java.base/java.lang.Thread.run(Thread.java:834)

mark-piller · April 10, 2020, 5:05pm

@Mario_Canevarollo, what is your application ID?

Mario_Canevarollo · April 10, 2020, 5:30pm

Hi Mark! 9171DE0B-E8E1-389D-FF48-60C0FD002A00

mark-piller · April 10, 2020, 6:10pm

Hi Mario,

Here’s what I just tried:

Ran an export of your Deals table.
Imported the produced csv file (Deals.csv) into my app. This recreated your Deals table in my app.
Deleted Deals.csv
Renamed test.csv which Andriy shared with you to Deals.csv
Imported Deals.csvinto my app
No problems were reported.

Could you please try the same in your app? Basically here are the steps:

Run an export of your Deals table.
Import the produced csv file (Deals.csv) into your app. This recreates your Deals table in the app.
Delete Deals.csv
Rename test.csv which Andriy shared with you to Deals.csv
Import Deals.csvinto your app. See if there are any import errors.

Regards,
Mark

mark-piller · April 10, 2020, 7:15pm

@Mario_Canevarollo In my instructions for you, right after step 1, I forgot to say to delete your existing Deals table. The table will be recreated by the import… I am sorry for missing this important detail.

Mario_Canevarollo · April 10, 2020, 7:43pm

WONDERFUL!!! It is solved, thanks a lot guys!!! The timestamp was the problem… really good @Andriy_Konoz !
Have a nice easter bro!