-
Notifications
You must be signed in to change notification settings - Fork 13.3k
[TableGen] Only store direct superclasses in Record #123072
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
@llvm/pr-subscribers-tablegen Author: Jay Foad (jayfoad) ChangesIn Record only store the direct superclasses instead of all This gives a small reduction in memory usage. On lib/Target/X86/X86.td I Full diff: https://github.com/llvm/llvm-project/pull/123072.diff 10 Files Affected:
diff --git a/llvm/docs/TableGen/BackGuide.rst b/llvm/docs/TableGen/BackGuide.rst
index c26575272ad692..52a02e0b85b3ba 100644
--- a/llvm/docs/TableGen/BackGuide.rst
+++ b/llvm/docs/TableGen/BackGuide.rst
@@ -610,29 +610,29 @@ functions returns null.
Getting Record Superclasses
===========================
-The ``Record`` class provides a function to obtain the superclasses of a
-record. It is named ``getSuperClasses`` and returns an ``ArrayRef`` of an
-array of ``std::pair`` pairs. The superclasses are in post-order: the order
-in which the superclasses were visited while copying their fields into the
-record. Each pair consists of a pointer to the ``Record`` instance for a
-superclass record and an instance of the ``SMRange`` class. The range
-indicates the source file locations of the beginning and end of the class
-definition.
+The ``Record`` class provides a function to obtain the direct superclasses
+of a record. It is named ``getDirectSuperClasses`` and returns an
+``ArrayRef`` of an array of ``std::pair`` pairs. Each pair consists of a
+pointer to the ``Record`` instance for a superclass record and an instance
+of the ``SMRange`` class. The range indicates the source file locations of
+the beginning and end of the class definition.
-This example obtains the superclasses of the ``Prototype`` record and then
-iterates over the pairs in the returned array.
+This example obtains the direct superclasses of the ``Prototype`` record and
+then iterates over the pairs in the returned array.
.. code-block:: text
ArrayRef<std::pair<const Record *, SMRange>>
- Superclasses = Prototype->getSuperClasses();
- for (const auto &SuperPair : Superclasses) {
+ Superclasses = Prototype->getDirectSuperClasses();
+ for (const auto &[Super, Range] : Superclasses) {
...
}
-The ``Record`` class also provides a function, ``getDirectSuperClasses``, to
-append the *direct* superclasses of a record to a given vector of type
-``SmallVectorImpl<Record *>``.
+The ``Record`` class also provides a function, ``getSuperClasses``, to
+append *all* superclasses of a record to a given vector of type
+``SmallVectorImpl<Record *>``. The superclasses are in post-order: the order
+in which the superclasses were visited while copying their fields into the
+record.
Emitting Text to the Output Stream
==================================
diff --git a/llvm/include/llvm/TableGen/Record.h b/llvm/include/llvm/TableGen/Record.h
index b15e9fc7328da0..45da0af25894f6 100644
--- a/llvm/include/llvm/TableGen/Record.h
+++ b/llvm/include/llvm/TableGen/Record.h
@@ -1630,9 +1630,9 @@ class Record {
SmallVector<AssertionInfo, 0> Assertions;
SmallVector<DumpInfo, 0> Dumps;
- // All superclasses in the inheritance forest in post-order (yes, it
+ // Direct superclasses, which are roots of the inheritance forest (yes, it
// must be a forest; diamond-shaped inheritance is not allowed).
- SmallVector<std::pair<const Record *, SMRange>, 0> SuperClasses;
+ SmallVector<std::pair<const Record *, SMRange>, 0> DirectSuperClasses;
// Tracks Record instances. Not owned by Record.
RecordKeeper &TrackedRecords;
@@ -1666,8 +1666,9 @@ class Record {
Record(const Record &O)
: Name(O.Name), Locs(O.Locs), TemplateArgs(O.TemplateArgs),
Values(O.Values), Assertions(O.Assertions),
- SuperClasses(O.SuperClasses), TrackedRecords(O.TrackedRecords),
- ID(getNewUID(O.getRecords())), Kind(O.Kind) {}
+ DirectSuperClasses(O.DirectSuperClasses),
+ TrackedRecords(O.TrackedRecords), ID(getNewUID(O.getRecords())),
+ Kind(O.Kind) {}
static unsigned getNewUID(RecordKeeper &RK);
@@ -1718,15 +1719,23 @@ class Record {
ArrayRef<AssertionInfo> getAssertions() const { return Assertions; }
ArrayRef<DumpInfo> getDumps() const { return Dumps; }
- ArrayRef<std::pair<const Record *, SMRange>> getSuperClasses() const {
- return SuperClasses;
+ void getSuperClasses(
+ SmallVectorImpl<std::pair<const Record *, SMRange>> &Classes) const {
+ for (const auto &[SC, R] : DirectSuperClasses) {
+ SC->getSuperClasses(Classes);
+ Classes.emplace_back(SC, R);
+ }
}
/// Determine whether this record has the specified direct superclass.
- bool hasDirectSuperClass(const Record *SuperClass) const;
+ bool hasDirectSuperClass(const Record *SuperClass) const {
+ return is_contained(make_first_range(DirectSuperClasses), SuperClass);
+ }
- /// Append the direct superclasses of this record to Classes.
- void getDirectSuperClasses(SmallVectorImpl<const Record *> &Classes) const;
+ /// Return the direct superclasses of this record.
+ ArrayRef<std::pair<const Record *, SMRange>> getDirectSuperClasses() const {
+ return DirectSuperClasses;
+ }
bool isTemplateArg(const Init *Name) const {
return llvm::is_contained(TemplateArgs, Name);
@@ -1794,29 +1803,32 @@ class Record {
void checkUnusedTemplateArgs();
bool isSubClassOf(const Record *R) const {
- for (const auto &[SC, _] : SuperClasses)
- if (SC == R)
+ for (const auto &[SC, _] : DirectSuperClasses) {
+ if (SC == R || SC->isSubClassOf(R))
return true;
+ }
return false;
}
bool isSubClassOf(StringRef Name) const {
- for (const auto &[SC, _] : SuperClasses) {
+ for (const auto &[SC, _] : DirectSuperClasses) {
if (const auto *SI = dyn_cast<StringInit>(SC->getNameInit())) {
if (SI->getValue() == Name)
return true;
} else if (SC->getNameInitAsString() == Name) {
return true;
}
+ if (SC->isSubClassOf(Name))
+ return true;
}
return false;
}
- void addSuperClass(const Record *R, SMRange Range) {
+ void addDirectSuperClass(const Record *R, SMRange Range) {
assert(!CorrespondingDefInit &&
"changing type of record after it has been referenced");
assert(!isSubClassOf(R) && "Already subclassing record!");
- SuperClasses.push_back(std::make_pair(R, Range));
+ DirectSuperClasses.emplace_back(R, Range);
}
/// If there are any field references that refer to fields that have been
diff --git a/llvm/lib/TableGen/DetailedRecordsBackend.cpp b/llvm/lib/TableGen/DetailedRecordsBackend.cpp
index 4a337248c941a1..2a1623e64e439d 100644
--- a/llvm/lib/TableGen/DetailedRecordsBackend.cpp
+++ b/llvm/lib/TableGen/DetailedRecordsBackend.cpp
@@ -152,8 +152,8 @@ void DetailedRecordsEmitter::printTemplateArgs(const Record &Rec,
// are enclosed in parentheses.
void DetailedRecordsEmitter::printSuperclasses(const Record &Rec,
raw_ostream &OS) {
- ArrayRef<std::pair<const Record *, SMRange>> Superclasses =
- Rec.getSuperClasses();
+ SmallVector<std::pair<const Record *, SMRange>> Superclasses;
+ Rec.getSuperClasses(Superclasses);
if (Superclasses.empty()) {
OS << " Superclasses: (none)\n";
return;
diff --git a/llvm/lib/TableGen/JSONBackend.cpp b/llvm/lib/TableGen/JSONBackend.cpp
index d648019ac46e8d..fecd16764fca21 100644
--- a/llvm/lib/TableGen/JSONBackend.cpp
+++ b/llvm/lib/TableGen/JSONBackend.cpp
@@ -151,7 +151,9 @@ void JSONEmitter::run(raw_ostream &OS) {
json::Array SuperClasses;
// Add this def to the instance list for each of its superclasses.
- for (const auto &[SuperClass, Loc] : Def->getSuperClasses()) {
+ SmallVector<std::pair<const Record *, SMRange>> SCs;
+ Def->getSuperClasses(SCs);
+ for (const auto &[SuperClass, Loc] : SCs) {
std::string SuperName = SuperClass->getNameInitAsString();
SuperClasses.push_back(SuperName);
InstanceLists[SuperName].push_back(Name);
diff --git a/llvm/lib/TableGen/Record.cpp b/llvm/lib/TableGen/Record.cpp
index 597ccb7ca144bb..4f94b3a48f9123 100644
--- a/llvm/lib/TableGen/Record.cpp
+++ b/llvm/lib/TableGen/Record.cpp
@@ -333,7 +333,7 @@ static const RecordRecTy *resolveRecordTypes(const RecordRecTy *T1,
if (T2->isSubClassOf(R)) {
CommonSuperClasses.push_back(R);
} else {
- R->getDirectSuperClasses(Stack);
+ append_range(Stack, make_first_range(R->getDirectSuperClasses()));
}
}
@@ -2394,11 +2394,8 @@ const DefInit *VarDefInit::instantiate() {
NewRec->resolveReferences(R);
- // Add superclasses.
- for (const auto &[SC, Loc] : Class->getSuperClasses())
- NewRec->addSuperClass(SC, Loc);
-
- NewRec->addSuperClass(
+ // Add superclass.
+ NewRec->addDirectSuperClass(
Class, SMRange(Class->getLoc().back(), Class->getLoc().back()));
// Resolve internal references and store in record keeper
@@ -2874,7 +2871,7 @@ void Record::checkName() {
const RecordRecTy *Record::getType() const {
SmallVector<const Record *, 4> DirectSCs;
- getDirectSuperClasses(DirectSCs);
+ append_range(DirectSCs, make_first_range(getDirectSuperClasses()));
return RecordRecTy::get(TrackedRecords, DirectSCs);
}
@@ -2906,35 +2903,6 @@ void Record::setName(const Init *NewName) {
// this. See TGParser::ParseDef and TGParser::ParseDefm.
}
-// NOTE for the next two functions:
-// Superclasses are in post-order, so the final one is a direct
-// superclass. All of its transitive superclases immediately precede it,
-// so we can step through the direct superclasses in reverse order.
-
-bool Record::hasDirectSuperClass(const Record *Superclass) const {
- ArrayRef<std::pair<const Record *, SMRange>> SCs = getSuperClasses();
-
- for (int I = SCs.size() - 1; I >= 0; --I) {
- const Record *SC = SCs[I].first;
- if (SC == Superclass)
- return true;
- I -= SC->getSuperClasses().size();
- }
-
- return false;
-}
-
-void Record::getDirectSuperClasses(
- SmallVectorImpl<const Record *> &Classes) const {
- ArrayRef<std::pair<const Record *, SMRange>> SCs = getSuperClasses();
-
- while (!SCs.empty()) {
- const Record *SC = SCs.back().first;
- SCs = SCs.drop_back(1 + SC->getSuperClasses().size());
- Classes.push_back(SC);
- }
-}
-
void Record::resolveReferences(Resolver &R, const RecordVal *SkipVal) {
const Init *OldName = getNameInit();
const Init *NewName = Name->resolveReferences(R);
@@ -3008,7 +2976,8 @@ raw_ostream &llvm::operator<<(raw_ostream &OS, const Record &R) {
}
OS << " {";
- ArrayRef<std::pair<const Record *, SMRange>> SC = R.getSuperClasses();
+ SmallVector<std::pair<const Record *, SMRange>> SC;
+ R.getSuperClasses(SC);
if (!SC.empty()) {
OS << "\t//";
for (const auto &[SC, _] : SC)
diff --git a/llvm/lib/TableGen/SetTheory.cpp b/llvm/lib/TableGen/SetTheory.cpp
index ac7ae2cbaed57b..b3a2b9d0d6fa7e 100644
--- a/llvm/lib/TableGen/SetTheory.cpp
+++ b/llvm/lib/TableGen/SetTheory.cpp
@@ -312,7 +312,9 @@ const RecVec *SetTheory::expand(const Record *Set) {
return &I->second;
// This is the first time we see Set. Find a suitable expander.
- for (const auto &[SuperClass, Loc] : Set->getSuperClasses()) {
+ SmallVector<std::pair<const Record *, SMRange>> SCs;
+ Set->getSuperClasses(SCs);
+ for (const auto &[SuperClass, Loc] : SCs) {
// Skip unnamed superclasses.
if (!isa<StringInit>(SuperClass->getNameInit()))
continue;
diff --git a/llvm/lib/TableGen/TGParser.cpp b/llvm/lib/TableGen/TGParser.cpp
index 60ae11b7f42612..b15de794408c12 100644
--- a/llvm/lib/TableGen/TGParser.cpp
+++ b/llvm/lib/TableGen/TGParser.cpp
@@ -330,17 +330,10 @@ bool TGParser::AddSubClass(Record *CurRec, SubClassReference &SubClass) {
// Since everything went well, we can now set the "superclass" list for the
// current record.
- for (const auto &[SC, Loc] : SC->getSuperClasses()) {
- if (CurRec->isSubClassOf(SC))
- return Error(SubClass.RefRange.Start,
- "Already subclass of '" + SC->getName() + "'!\n");
- CurRec->addSuperClass(SC, Loc);
- }
-
if (CurRec->isSubClassOf(SC))
return Error(SubClass.RefRange.Start,
"Already subclass of '" + SC->getName() + "'!\n");
- CurRec->addSuperClass(SC, SubClass.RefRange);
+ CurRec->addDirectSuperClass(SC, SubClass.RefRange);
return false;
}
@@ -4003,7 +3996,7 @@ bool TGParser::ParseClass() {
if (CurRec) {
// If the body was previously defined, this is an error.
if (!CurRec->getValues().empty() ||
- !CurRec->getSuperClasses().empty() ||
+ !CurRec->getDirectSuperClasses().empty() ||
!CurRec->getTemplateArgs().empty())
return TokError("Class '" + CurRec->getNameInitAsString() +
"' already defined");
diff --git a/llvm/utils/TableGen/CallingConvEmitter.cpp b/llvm/utils/TableGen/CallingConvEmitter.cpp
index b315aa7d86fe91..ce97cbf17ddf8c 100644
--- a/llvm/utils/TableGen/CallingConvEmitter.cpp
+++ b/llvm/utils/TableGen/CallingConvEmitter.cpp
@@ -109,12 +109,13 @@ void CallingConvEmitter::emitCallingConv(const Record *CC, raw_ostream &O) {
// Emit all of the actions, in order.
for (unsigned I = 0, E = CCActions->size(); I != E; ++I) {
const Record *Action = CCActions->getElementAsRecord(I);
+ SmallVector<std::pair<const Record *, SMRange>> SCs;
+ Action->getSuperClasses(SCs);
SwiftAction =
- llvm::any_of(Action->getSuperClasses(),
- [](const std::pair<const Record *, SMRange> &Class) {
- std::string Name = Class.first->getNameInitAsString();
- return StringRef(Name).starts_with("CCIfSwift");
- });
+ llvm::any_of(SCs, [](const std::pair<const Record *, SMRange> &Class) {
+ std::string Name = Class.first->getNameInitAsString();
+ return StringRef(Name).starts_with("CCIfSwift");
+ });
O << "\n";
emitAction(Action, indent(2), O);
diff --git a/llvm/utils/TableGen/Common/CodeGenRegisters.cpp b/llvm/utils/TableGen/Common/CodeGenRegisters.cpp
index 2dbee94d7e5406..ed8891d704fba0 100644
--- a/llvm/utils/TableGen/Common/CodeGenRegisters.cpp
+++ b/llvm/utils/TableGen/Common/CodeGenRegisters.cpp
@@ -703,8 +703,8 @@ struct TupleExpander : SetTheory::Expander {
"Register tuple redefines register '" + Name + "'.");
// Copy Proto super-classes.
- for (const auto &[Super, Loc] : Proto->getSuperClasses())
- NewReg->addSuperClass(Super, Loc);
+ for (const auto &[Super, Loc] : Proto->getDirectSuperClasses())
+ NewReg->addDirectSuperClass(Super, Loc);
// Copy Proto fields.
for (unsigned i = 0, e = Proto->getValues().size(); i != e; ++i) {
diff --git a/llvm/utils/TableGen/SearchableTableEmitter.cpp b/llvm/utils/TableGen/SearchableTableEmitter.cpp
index 91fde0c6630577..caae0037de9956 100644
--- a/llvm/utils/TableGen/SearchableTableEmitter.cpp
+++ b/llvm/utils/TableGen/SearchableTableEmitter.cpp
@@ -839,7 +839,7 @@ void SearchableTableEmitter::run(raw_ostream &OS) {
const Record *SearchableTable = Records.getClass("SearchableTable");
for (auto &NameRec : Records.getClasses()) {
const Record *Class = NameRec.second.get();
- if (Class->getSuperClasses().size() != 1 ||
+ if (Class->getDirectSuperClasses().size() != 1 ||
!Class->isSubClassOf(SearchableTable))
continue;
|
This will also need some updates in Clang and MLIR. |
I just updated the description because the memory savings are noticeably better after 2eb1354. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Waiting for clang/mlir changes, otherwise LGTM.
of the ``SMRange`` class. The range indicates the source file locations of | ||
the beginning and end of the class definition. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not related to this PR, but since you are changing the comment:
The range indicates the source file locations of the beginning and end of the class definition.
Is this true? TGParser::AddSubClass
seems to pass the location of a superclass reference in the inheritance list.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I noticed this too, and I think you're probably right and the documentation is wrong, but I didn't want to go down the rabbit hole of fact-checking the documentation just now.
llvm/include/llvm/TableGen/Record.h
Outdated
return SuperClasses; | ||
/// Append all superclasses in post-order to \p Classes. | ||
void getSuperClasses(std::vector<const Record *> &Classes) const { | ||
for (const auto &[SC, R] : DirectSuperClasses) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use _
instead of R
like you did in isSubClassOf
?
Or can we use make_first_range
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I went for make_first_range
. Done.
/// Append the direct superclasses of this record to Classes. | ||
void getDirectSuperClasses(SmallVectorImpl<const Record *> &Classes) const; | ||
/// Return the direct superclasses of this record. | ||
ArrayRef<std::pair<const Record *, SMRange>> getDirectSuperClasses() const { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(idea for a future patch)
It looks like no clients currently make use of the returned SMRange
. It may make sense to provide two methods getDirectSuperClasses
returning records and getDirectSuperClassReferenceLocs
returning superclass reference locations. (This would require storing records/locations separately or some map_range
magic).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like no clients currently make use of the returned
SMRange
.
CodeGenRegisters
uses it, but it looks like it is just trying to clone a Record, and there may be better ways to reimplement that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like no clients currently make use of the returned
SMRange
.
Oh I see, it is copied around but ultimately never used for anything. I can do a follow up to remove it completely.
Or maybe doing that change first, before landing this one, would be less disruptive (in case there are any out-of-tree users of getSuperClasses
/getDirectSuperClasses
)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure location should be removed completely as it might be used for diagnostics.
On the other hand, I can't immediately think of a case where the location of a super-class reference in an error message would be useful.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure location should be removed completely as it might be used for diagnostics.
On the other hand, I can't immediately think of a case where the location of a super-class reference in an error message would be useful.
Though unfortunately we're not doing it at this moment, I think we should make good uses of this location (where the superclass was reference) to show a nice backtrace. Personally I feel like if it doesn't take a lot of efforts we should probably keep the location.
All done now. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
/// Append the direct superclasses of this record to Classes. | ||
void getDirectSuperClasses(SmallVectorImpl<const Record *> &Classes) const; | ||
/// Return the direct superclasses of this record. | ||
ArrayRef<std::pair<const Record *, SMRange>> getDirectSuperClasses() const { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure location should be removed completely as it might be used for diagnostics.
On the other hand, I can't immediately think of a case where the location of a super-class reference in an error message would be useful.
Though unfortunately we're not doing it at this moment, I think we should make good uses of this location (where the superclass was reference) to show a nice backtrace. Personally I feel like if it doesn't take a lot of efforts we should probably keep the location.
// must be a forest; diamond-shaped inheritance is not allowed). | ||
SmallVector<std::pair<const Record *, SMRange>, 0> SuperClasses; | ||
SmallVector<std::pair<const Record *, SMRange>, 0> DirectSuperClasses; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given the fact that we're only storing a small number of direct super classes now, would it make more sense to use SmallVector<..., N> with N not equal to zero? N = 0 will disable any stack allocation and allocate on heap only, IIRC
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried N=1 but it seemed to increase memory usage overall. I think the problem is that Record
is used for too many different things, some of which have superclasses and some do not. I would much prefer that we use more specialized types. Most importantly I think "static" objects like class
statements and def
statements should be represented differently from "runtime" objects like instantiated defs.
Co-authored-by: Min-Yih Hsu <min@myhsu.dev>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
@jayfoad reverse-ping - what's the plan for this please? |
To forget about it :) I think it's good to go now. |
In Record only store the direct superclasses instead of all
superclasses. getSuperClasses recurses to find all superclasses when
necessary.
This gives a small reduction in memory usage. On lib/Target/X86/X86.td I
measured about 2.0% reduction in total bytes allocated (measured by
valgrind) and 1.3% reduction in peak memory usage (measured by
/usr/bin/time -v).