Variables from SPSS SAV are left out

[zenelba]

Hi!
This one happened to me for the first time - at least one variable that exists in SPSS SAV file was not imported into pandas dataframe.
I am using the latest version of the library.

I am uploading the database. The variable that is missing is (at least) XC0DAB1_1.

What could be the reason?

Happy New Year!

z
CRO_MX.zip

[ofajardo]

happy new year!

The variable XC0DAB1_1 is in position 85 when I open it using pspp. When reading it in pyreadstat, in position 85 (here 84 since python starts numbering at 0) I see the variable XC0DAB1. Also looking at the variables 84 and 86, they seem to match what I see in pspp, and the data itself matches pspp, so my conclusion is that the variable is there but the name has been truncated. This is probable the same bug as #165. This bug has been reported upstream for fixing, but unfortunately there is no solution yet.

>>> import pyreadstat as p
>>> f = "CRO_MX.SAV"
>>> df, meta = p.read_sav(f)
>>> col = df.columns.to_list()
# name truncated in variable #85 
>>> col[84]
'XC0DAB1'
# the variables next to it match what I see in pspp
>>> col[85]
'XC0DAB1_2'
>>> col[83]
'XC0DABS1_list_1'
# the data also matches what I see in pspp
>>> df.iloc[:, 83:86]
  XC0DABS1_list_1    XC0DAB1  XC0DAB1_2
0          Vegeta      999.0        NaN
1          Vegeta  1000000.0        NaN

[ofajardo]

I have added your file to the bug report in Readstat (see WizardMac/ReadStat#268).

Maybe the reporter of the original bug has any workaround? @Ov-ille, any suggestions?

[zenelba]

happy new year!

Same to you!
And thanks for looking at this case.

variable is there but the name has been truncated.

So the solution is not to use _ in namings. What happens often :(

The problem is that I have plenty of _ in variable names. And this is the first time it happened. After few years of intensive use.

Will try to save it in another format than sav as inout and produce labels differently.