Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bytewise read() of files is slow #146

Closed
hjelmeland opened this issue Nov 2, 2020 · 3 comments
Closed

Bytewise read() of files is slow #146

hjelmeland opened this issue Nov 2, 2020 · 3 comments

Comments

@hjelmeland
Copy link

By changing to buffered read in yang_parse_file(), I reduced the startup time of clixon_backend to a fifth. In the test I loaded many large YANG models, but only a small data store. I only tested on PC for now, getting a speed up of 200 ms. On a slow target, I suspect improvement can be seconds.

For the test I just did a fp=fdopen(fd, "r"); and then replaced the read with fread( &c, 1, 1, fp)). I think it would be best to change the API so that yang_parse_file(), clixon_xml_parse_file() and clixon_json_parse_file() gets a FILE* instead of a int file descriptor. But I don't know if you like to change the API.

@olofhagsand
Copy link
Member

Thanks for looking into this. I need to look at the API.

@olofhagsand
Copy link
Member

Confirm it gives ~16 x improvement on large xml files.
I see it as doable as changing the API, these are not typically exposed functions, at least not in the application I have insight.

olofhagsand added a commit that referenced this issue Nov 4, 2020
… functions:

  * clixon_xml_parse_file(), clixon_json_parse_file(), yang_parse_file()
  * See [Bytewise read() of files is slow #146](#146)
@olofhagsand
Copy link
Member

Fixed by patch ^

olofhagsand added a commit that referenced this issue Nov 5, 2020
… functions:

  * clixon_xml_parse_file(), clixon_json_parse_file(), yang_parse_file()
  * See [Bytewise read() of files is slow #146](#146)
olofhagsand added a commit that referenced this issue Nov 9, 2020
… functions:

  * clixon_xml_parse_file(), clixon_json_parse_file(), yang_parse_file()
  * See [Bytewise read() of files is slow #146](#146)
olofhagsand added a commit that referenced this issue Aug 4, 2021
… functions:

  * clixon_xml_parse_file(), clixon_json_parse_file(), yang_parse_file()
  * See [Bytewise read() of files is slow #146](#146)
olofhagsand added a commit that referenced this issue Aug 4, 2021
… functions:

  * clixon_xml_parse_file(), clixon_json_parse_file(), yang_parse_file()
  * See [Bytewise read() of files is slow #146](#146)
olofhagsand added a commit that referenced this issue Aug 17, 2021
… functions:

  * clixon_xml_parse_file(), clixon_json_parse_file(), yang_parse_file()
  * See [Bytewise read() of files is slow #146](#146)
olofhagsand added a commit that referenced this issue Aug 17, 2021
… functions:

  * clixon_xml_parse_file(), clixon_json_parse_file(), yang_parse_file()
  * See [Bytewise read() of files is slow #146](#146)
olofhagsand added a commit that referenced this issue Aug 31, 2021
… functions:

  * clixon_xml_parse_file(), clixon_json_parse_file(), yang_parse_file()
  * See [Bytewise read() of files is slow #146](#146)
olofhagsand added a commit that referenced this issue Aug 31, 2021
… functions:

  * clixon_xml_parse_file(), clixon_json_parse_file(), yang_parse_file()
  * See [Bytewise read() of files is slow #146](#146)
olofhagsand added a commit that referenced this issue Sep 21, 2021
… functions:

  * clixon_xml_parse_file(), clixon_json_parse_file(), yang_parse_file()
  * See [Bytewise read() of files is slow #146](#146)
olofhagsand added a commit that referenced this issue Sep 21, 2021
… functions:

  * clixon_xml_parse_file(), clixon_json_parse_file(), yang_parse_file()
  * See [Bytewise read() of files is slow #146](#146)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants