Repository navigation
Add tools for generating mappings_XX.h #84508
Description
| BPO | 40328 |
|---|---|
| Nosy | @hyeshik, @vstinner, @methane, @serhiy-storchaka, @corona10 |
| PRs |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2020-04-19.13:13:59.400>
labels = ['3.9']
title = 'Add tools for generating mappings_XX.h'
updated_at = <Date 2020-05-25.04:14:49.537>
user = 'https://github.com/corona10'bugs.python.org fields:
activity = <Date 2020-05-25.04:14:49.537>
actor = 'corona10'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Demos and Tools']
creation = <Date 2020-04-19.13:13:59.400>
creator = 'corona10'
dependencies = []
files = []
hgrepos = []
issue_num = 40328
keywords = ['patch']
message_count = 4.0
messages = ['366777', '367682', '369844', '369845']
nosy_count = 5.0
nosy_names = ['hyeshik.chang', 'vstinner', 'methane', 'serhiy.storchaka', 'corona10']
pr_nums = ['19602']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue40328'
versions = ['Python 3.9']
Activity
corona10 commentedon Apr 19, 2020
AFAIK, there are no tools for generating mappings_XX.h under Modules/cjkcodecs.
It would cause a maintainable issue,
fortunately, I got the old source from https://github.com/BackupTheBerlios/cjkpython.
And I success to port into Python3 version and successfully reproduce to generating header file except for mappings_hk.h mappings_tw.h.
The patch will include a generator and Unicode mapping file.
I am not the codec expert but hopefully it will help us.
corona10 commentedon Apr 29, 2020
New changeset 113feb3 by Dong-hee Na in branch 'master':
bpo-40328: Add tool for generating cjk mapping headers (GH-19602)
113feb3
methane commentedon May 25, 2020
Would you close this issue?
corona10 commentedon May 25, 2020
mappings_hk.h mappings_tw.h are still left :)
sorcio commentedon May 4, 2022
Hey @corona10, I happened upon this for other reasons. I found that Python's big5hkscs codec does not support the same character set that modern web browsers support.
There has been at least one version of HKSCS (2008) that updated the Big5-encoded character set, and one more recent version (2016) which only added characters to Unicode (with no Big5 mapping), but might adjust some of the existing mappings. I still need to understand if this might break compatibility.
Is it okay if I look into this and try to make a PR later? I've already done something for mappings_hk but need to look into the compatibility, so it might take a second.
corona10 commentedon May 11, 2022
@sorcio Oh thanks, PR will be welcomed :)
pythongh-84508: tool to generate cjk traditional chinese mappings
gh-84508: Add mapping files for Korean and Japanese. (gh-93309)
Revert "gh-84508: Add mapping files for Korean and Japanese. (gh-93309)"
Revert "gh-84508: Add mapping files for Korean and Japanese. (gh-93309)…
gh-84508: tool to generate cjk traditional chinese mappings (gh-93272)
Revert 30935 defer bpo45162 to 312 (#94285)