VimAssistant is a AI powered productivity assistant to VimKit that helps users complete tasks more efficiently.
The VimAssistant package processes natural language (spoken or typed) to take action on a VIM model.
This package utilizes Speech Recognition to transcribe spoken text that gets sent to a CoreML LLM/LAM that will attempt to execute an action on the users behalf.
Some examples of categorized actions include (but not limited to):
- ISOLATE: "Isolate furniture", "Only show me the network jacks", "Filter everything but the curtain panels"
- HIDE: "Conceal all ceiling systems", "Hide air terminals", "Remove all walls"
- QUANTIFY: "Count all doors", "What are the total number of access doors?", "How many transformers are there?"
The base model was created with the OntoNotes 5.0 NER annotations which includes:
- CARDINAL: Cardinal numbers (e.g., 1, 2, 3).
- DATE: Dates (e.g., May 8, 2025).
- EVENT: Names of events (e.g., World Series).
- FAC: Buildings or facilities (e.g., White House).
- GPE: Geo-political entities (e.g., United States).
- LANGUAGE: Names of languages (e.g., English).
- LAW: Legal names (e.g., The Constitution).
- LOC: Represents locations (e.g., "New York City").
- MONEY: Indicates monetary values (e.g., "100 dollars").
- NORP: Represents national or political or religious groups (e.g., "Democrats", "the Catholic Church").
- ORDINAL: Denotes ordinal numbers (e.g., "first", "second", "10th").
- ORG: Represents organizations (e.g., "Google", "Microsoft").
- PERCENT: Denotes percentages (e.g., "10%", "20%").
- PERSON: Individual names (e.g., Barack Obama).
- PRODUCT: Represents products (e.g., "iPhone", "MacBook").
- QUANTITY: Indicates measurements or quantities (e.g., "10 kilograms").
- TIME: Times (e.g., 10:00 AM).
- WORK_OF_ART: Names of works of art (e.g., "Hamlet").
The trained model provides Construction NER annotations:
- CON_BIM_CATG: BIM Category - a high-level classification for families and elements, grouping them based on their functional type.
- CON_BIM_FAML: BIM Family - a collection of elements that share common properties, behaviors, and physical characteristics.
- CON_BIM_TYPE: BIM Type - a specific instantiation of a family that defines a unique set of parameters, essentially a variation within a family. Think of it as a specific size, material, or configuration of a particular family, such as a 3' x 6' door within a door family.
- CON_BIM_INST: BIM Instance - a single, unique occurrence of a family type placed within a model.
- CON_BIM_LEVL: BIM Level - a horizontal plane used to define the vertical position of elements like walls, floors, and ceilings.
- CON_BIM_VIEW: BIM View - represents a specific way of looking at the model, whether it's a 2D plan, elevation, section, or 3D view.
| Component | Labels |
|---|---|
| named entities | CARDINAL, DATE, EVENT, FAC, GPE, LANGUAGE, LAW, LOC, MONEY, NORP, ORDINAL, ORG, PERCENT, PERSON, PRODUCT, QUANTITY, TIME, WORK_OF_ART, CON_BIM_CATG, CON_BIM_FAML, CON_BIM_TYPE, CON_BIM_INST, CON_BIM_LEVL, CON_BIM_VIEW |
| categories | ISOLATE, HIDE, QUANTIFY, ZOOM_IN, ZOOM_OUT, PAN_LEFT, PAN_RIGHT, PAN_UP, PAN_DOWN, LOOK_LEFT, LOOK_RIGHT, LOOK_UP, LOOK_DOWN |