Skip to content

haven is limited in how many columns it will write to a SAS7BDAT file #335

Closed

Description

Before launching into my code, I do want to say that I really love the tidyverse and I work with it every day. You've made my life easier and I appreciate that.

I tried to use haven to write really wide dataset, and it failed. Here's a reproducible example.

## create a wide data frame with N rows
NR=1000 # number of rows
NC=20000 # number of columns
wide_example=data.frame(matrix(1:(NR*NC),nrow=NR))
library(haven)
write_sas(wide_example,"some_file.sas7bdat")

Fails with this error message

Error in write_sas_(data, normalizePath(path, mustWork = FALSE)) : 
  Writing failure: A row of data will not fit into the file format.

The system craps out between 250 and 400 rows. I would imagine it also varies by the type of data in each row, but I'm less interested in the particulars than in the fact that there is a limit. SAS itself does not impose such a limit. Is there a PAGESIZE parameter that can be tuned? Now, granted, I don't expect the package to be able to create any SAS file, or it would be SAS!. But if there are limits, I would like to see them documented somewhere.

Note

The following SAS code creates the same data set with variables X1 through X20000 (more or less) and the output shows the page size.

%let nr=1000;
%let nc=20000;
data wide_test;
drop row col xval;
  xval=1;
  array x{&nc.};
  do row=1 to &nr.;
    do col=1 to &nc.;
	  x{col}=xval;
	  xval=xval+1;
	end;
	output;
  end;
run;

proc contents data=wide_test;
run;

Wide_test.pdf

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    bugan unexpected problem or unintended behaviorreadstat

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions