Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix DFile name parsing bug #8794

Merged
merged 2 commits into from
Oct 24, 2022

Conversation

haoyifan
Copy link

@haoyifan haoyifan commented Nov 25, 2021

When parsing DFile name, we expect the file name to follow the
format of v(vid)f(fid).(type)-ver(version). When parsing type
in tsdbParseDFilename(), it uses a glibc modifier "m" to allocate
string. However, type string ("p" variable in the original code)
could be NULL for various reasons, and the following code doesn't
check NULL, which would lead to crash.

This patch fixes this bug by using a stack allocated string to
parse type.

  • Testing

I have a local vnode repository like following:
.
├── vnode145
│   ├── config.json
│   ├── tsdb
│   │   ├── data
│   │   │   ├── v145f1736.data
│   │   │   ├── v145f1736.head-ver14
│   │   │   ├── v145f1736.last
│   │   │   ├── v145f1736.smad
│   │   │   └── v145f1736.smal
│   │   └── meta
│   ├── version.json
│   └── wal

"vnode145" is lacking "current" for the FS status, but this is fine
since it can be recovered from the "data" folder. However, the above
repository will lead to taosd crashing while restoring "current",
because type for DFile can sometimes be not parsed by the sscanf
format before this patch.

After this patch, the above repository can work OK and "current" can
be restored.

Also run taosdemo for sanity check.

Spent 81.9563 seconds to insert rows: 100000000, affected rows: 100000000 with 8 thread(s) into test.meters. 1220162.42 records/second

insert delay, avg: 65.13ms, max: 252.73ms, min: 24.77ms

Yifan Hao added 2 commits November 24, 2021 23:14
When parsing DFile name, we expect the file name to follow the
format of v<vid>f<fid>.<type>-ver<version>. When parsing type
in tsdbParseDFilename(), it uses a glibc modifier "m" to allocate
string. However, type string ("p" variable in the original code)
could be NULL for various reasons, and the following code doesn't
check NULL, while would lead to crash.

This patch fixes this bug by using a stack allocated string to
parse type.

* Testing

I have a local vnode repository like following:
.
├── vnode145
│   ├── config.json
│   ├── tsdb
│   │   ├── data
│   │   │   ├── v145f1736.data
│   │   │   ├── v145f1736.head-ver14
│   │   │   ├── v145f1736.last
│   │   │   ├── v145f1736.smad
│   │   │   └── v145f1736.smal
│   │   └── meta
│   ├── version.json
│   └── wal

"vnode145" is lacking "current" for the FS status, but this is fine
since it can be recovered from the "data" folder. However, the above
repository will lead to taosd crashing while restoring "current",
because type for DFile can sometimes be not parsed by the sscanf
format before this patch.

After this patch, the above repository can work OK and "current" can
be restored.

Also run taosdemo for sanity check.

Spent 81.9563 seconds to insert rows: 100000000, affected rows: 100000000 with 8 thread(s) into test.meters. 1220162.42 records/second

insert delay, avg:      65.13ms, max:     252.73ms, min:      24.77ms
Looks like Jenkins compiler is not happy with variable length
static array...

* Testing
Local compile pass

>gcc --version
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
Apple clang version 13.0.0 (clang-1300.0.29.3)
Target: x86_64-apple-darwin21.1.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

./taosdemo for sanity check
@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


Yifan Hao seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@guanshengliang guanshengliang merged commit fb6a8b1 into taosdata:develop Oct 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants