Activation collection to support probe training; iit training; relati…

…ve import path; more tutorials (stanfordnlp#24,stanfordnlp#26,stanfordnlp#40)

CONTRIBUTING.md

@@ -0,0 +1,51 @@
+# Contributing Guidelines
+
+*Pull requests, bug reports, and all other forms of contribution are welcomed and highly encouraged!* :octocat:
+
+### :book: Pull Requests
+
+#### Uninstall pyvene from python library
+It becomes tricky if you have `pyvene` installed while debugging with this codebase, since imports can be easily messed up. Please run,
+```bash
+pip uninstall pyvene
+```
+
+#### Unit Test Run Is A Must before Creating PRs
+When adding new methods or APIs, unit tests are now enforced. To run existing tests, you can kick off the python unittest command in the discovery mode as,
+```bash
+cd pyvene
+python -m unittest discover -p '*TestCase.py'
+```
+When checking in new code, please also consider to add new tests in the same PR. Please include test results in the PR to make sure all the existing test cases are passing. Please see the `qa_runbook.ipynb` notebook about a set of conventions about how to add test cases. The code coverage for this repository is currently `low`, and we are adding more automated tests.
+
+#### Format
+```
+**Descriptions**:
+
+[Describe your PR Here]
+
+
+**Testing Done**:
+
+[Provide logs, screen-shots, files that contain tests you have done]
+
+```
+
+Go to issues, and open with a title formatted as,
+```
+[Priority Tag] Short Title
+```
+For Priority Tag, you can use P0-P2, P0 is the highest priority, which means everyone should stop working and focus on this PR.
+
+### :beetle: Bug Reports and Other Issues
+Go to issues, and open with a title formatted as,
+```
+[Bug Fix] Short Title
+```
+For external requests (i.e., you are not in our core dev team), please use,
+```
+[External] Short Title
+```
+
+### :inbox_tray: Larger Feature Requests
+Please email us!

README.md

@@ -16,11 +16,22 @@ Install with pip on stable releases,
 ```bash
 pip install pyvene
 ```
+
 or with our dev repo directly,
 ```bash
 pip install git+https://github.com/frankaging/pyvene.git
 ```
 
+or you can clone our repo,
+```bash
+git clone https://github.com/frankaging/pyvene.git
+```
+and import to your project as,
+```python
+from pyvene import pyvene
+_, tokenizer, gpt2 = pyvene.create_gpt2()
+```
+
 ## _Wrap_ , _Intervene_ and _Share_
 You can intervene with supported models as,
 ```python
@@ -142,13 +153,10 @@ intervenable.train(
 where you need to pass in a trainable dataset, and your customized loss and metrics function. The trainable interventions can later be saved on to your disk. You can also use `intervenable.evaluate()` your interventions in terms of customized objectives.
 
 
-## Unit-tests
-When adding new methods or APIs, unit tests are now enforced. To run existing tests, you can kick off the python unittest command in the discovery mode as,
-```bash
-cd pyvene
-python -m unittest discover -p '*TestCase.py'
-```
-When checking in new code, please also consider to add new tests in the same PR. Please include test results in the PR to make sure all the existing test cases are passing. Please see the `qa_runbook.ipynb` notebook about a set of conventions about how to add test cases. The code coverage for this repository is currently `low`, and we are adding more automated tests.
+## Contributing to This Library
+Please see [our guidelines](CONTRIBUTING.md) about how to contribute to this repository.         
+
+*Pull requests, bug reports, and all other forms of contribution are welcomed and highly encouraged!* :octocat:  
 
 
 ## Related Works in Discovering Causal Mechanism of LLMs

pyvene/__init__.py

@@ -19,6 +19,7 @@
 from .models.interventions import SigmoidMaskRotatedSpaceIntervention
 from .models.interventions import LowRankRotatedSpaceIntervention
 from .models.interventions import PCARotatedSpaceIntervention
+from .models.interventions import CollectIntervention
 
 
 # Utils

pyvene/models/blip/modelings_intervenable_blip.py

@@ -9,7 +9,7 @@
 """
 
 
-from pyvene.models.constants import CONST_INPUT_HOOK, CONST_OUTPUT_HOOK, CONST_QKV_INDICES
+from ..constants import CONST_INPUT_HOOK, CONST_OUTPUT_HOOK, CONST_QKV_INDICES
 
 """blip base model"""
 blip_type_to_module_mapping = {

pyvene/models/configuration_intervenable_model.py

@@ -5,7 +5,7 @@
 from transformers import PreTrainedTokenizer, TensorType, is_torch_available
 from transformers.configuration_utils import PretrainedConfig
 
-from pyvene.models.interventions import VanillaIntervention
+from .interventions import VanillaIntervention
 
 
 IntervenableRepresentationConfig = namedtuple(

pyvene/models/gpt2/modelings_intervenable_gpt2.py

@@ -9,7 +9,7 @@
 """
 
 
-from pyvene.models.constants import CONST_INPUT_HOOK, CONST_OUTPUT_HOOK, CONST_QKV_INDICES
+from ..constants import CONST_INPUT_HOOK, CONST_OUTPUT_HOOK, CONST_QKV_INDICES
 
 
 """gpt2 base model"""

pyvene/models/gpt_neo/modelings_intervenable_gpt_neo.py

@@ -9,7 +9,7 @@
 """
 
 
-from pyvene.models.constants import CONST_INPUT_HOOK, CONST_OUTPUT_HOOK, CONST_QKV_INDICES
+from ..constants import CONST_INPUT_HOOK, CONST_OUTPUT_HOOK, CONST_QKV_INDICES
 
 
 """gpt_neo base model"""

pyvene/models/gpt_neox/modelings_intervenable_gpt_neox.py

@@ -9,7 +9,7 @@
 """
 
 
-from pyvene.models.constants import CONST_INPUT_HOOK, CONST_OUTPUT_HOOK, CONST_QKV_INDICES
+from ..constants import CONST_INPUT_HOOK, CONST_OUTPUT_HOOK, CONST_QKV_INDICES
 
 
 """gpt_neox base model"""

pyvene/models/gru/modelings_intervenable_gru.py

@@ -9,7 +9,7 @@
 """
 
 
-from pyvene.models.constants import CONST_INPUT_HOOK, CONST_OUTPUT_HOOK
+from ..constants import CONST_INPUT_HOOK, CONST_OUTPUT_HOOK
 
 
 """gru base model"""