vpkpp: add support for ORE (Narbacular Drop) pack files

FUTURE.md

@@ -6,6 +6,10 @@
 
 ### Future Improvements
 
+- External
+  - Move from cryptopp to a maintained cryptography library
+  - Move from compressonator to something that isn't a pile of garbage
+  - Fork minizip-ng and change how it handles dependencies
 - `bsppp`
   - Read/write LZMA-compressed lumps
 - `dmxpp`
@@ -17,6 +21,10 @@
     - When creating an instance of the game, attaching a console might be easier, or enabling -condebug,
       which would then allow reading output from the console
   - Add a method to take a screenshot of the game and move the file to a user-specified location
+- `kvpp`
+  - Reduce template spam
+  - Move DMX/KV2 code here
+  - Add a KV3 parser/writer
 - `mdlpp`
   - Parse animations/sequences
   - Make something to construct StudioModel objects from a given model file like obj/glTF?
@@ -26,6 +34,7 @@
 - `vcryptpp`
   - Add VFONT write support
 - `vpkpp`
+  - Add write support to ORE
   - Cache file handles so they're not constantly getting closed/opened
 - `vtfpp`
   - Allow directly applying an HDRI to a VTF without requiring the user to write a custom wrapper

README.md

@@ -129,11 +129,11 @@ Several modern C++20 libraries for sanely parsing Valve formats, rolled into one
   </tr>
   <tr><!-- empty row to disable github striped bg color --></tr>
   <tr>
-    <td rowspan="25"><code>vpkpp</code></td>
+    <td rowspan="27"><code>vpkpp</code></td>
     <td><a href="https://developer.valvesoftware.com/wiki/Bonus_Maps">BMZ</a></td>
     <td align="center">✅</td>
     <td align="center">✅</td>
-    <td rowspan="25" align="center">C<br>C#</td>
+    <td rowspan="27" align="center">C<br>C#</td>
   </tr>
   <tr><!-- empty row to disable github striped bg color --></tr>
   <tr>
@@ -160,6 +160,12 @@ Several modern C++20 libraries for sanely parsing Valve formats, rolled into one
     <td align="center">✅</td>
   </tr>
   <tr><!-- empty row to disable github striped bg color --></tr>
+  <tr>
+    <td>ORE (Narbacular Drop)</td>
+    <td align="center">✅</td>
+    <td align="center">❌</td>
+  </tr>
+  <tr><!-- empty row to disable github striped bg color --></tr>
   <tr>
     <td><a href="https://quakewiki.org/wiki/.pak">PAK</a> (Quake, WON Half-Life)</td>
     <td align="center">✅</td>
@@ -305,7 +311,8 @@ found on PyPI in the [sourcepp](https://pypi.org/project/sourcepp) package.
 
 - `steampp` is based on the [SteamAppPathProvider](https://github.com/Trico-Everfire/SteamAppPathProvider) library by [@Trico Everfire](https://github.com/Trico-Everfire) and [Momentum Mod](https://momentum-mod.org) contributors.
 - `vpkpp`'s GCF parser was contributed by [@bt](https://github.com/caatge) and [@ymgve](https://github.com/ymgve).
-- `vpkpp`'s WAD3 parser was contributed by [@ozxybox](https://github.com/ozxybox).
+- `vpkpp`'s ORE parser is based on [reverse-engineering work](https://github.com/erysdren/narbacular-drop-tools) by [@erysdren](https://github.com/erysdren).
+- `vpkpp`'s WAD3 parser/writer was contributed by [@ozxybox](https://github.com/ozxybox).
 - `vtfpp`'s write support is loosely based on work by [@Trico Everfire](https://github.com/Trico-Everfire).
 - `vtfpp`'s HDRI to cubemap conversion code is modified from the [HdriToCubemap](https://github.com/ivarout/HdriToCubemap) library by [@ivarout](https://github.com/ivarout).
 

docs/index.md

@@ -115,11 +115,11 @@ Several modern C++20 libraries for sanely parsing Valve formats, rolled into one
     <td align="center">❌</td>
   </tr>
   <tr>
-    <td rowspan="13"><code>vpkpp</code></td>
+    <td rowspan="14"><code>vpkpp</code></td>
     <td><a href="https://developer.valvesoftware.com/wiki/Bonus_Maps">BMZ</a></td>
     <td align="center">✅</td>
     <td align="center">✅</td>
-    <td rowspan="13" align="center">C<br>C#</td>
+    <td rowspan="14" align="center">C<br>C#</td>
   </tr>
   <tr>
     <td><a href="https://developer.valvesoftware.com/wiki/BSP_(Source)">BSP</a> v17-27</td>
@@ -141,6 +141,11 @@ Several modern C++20 libraries for sanely parsing Valve formats, rolled into one
     <td align="center">✅</td>
     <td align="center">✅</td>
   </tr>
+  <tr>
+    <td>ORE (Narbacular Drop)</td>
+    <td align="center">✅</td>
+    <td align="center">❌</td>
+  </tr>
   <tr>
     <td><a href="https://quakewiki.org/wiki/.pak">PAK</a> (Quake, WON Half-Life)</td>
     <td align="center">✅</td>
@@ -268,7 +273,8 @@ found on PyPI in the [sourcepp](https://pypi.org/project/sourcepp) package.
 
 - `steampp` is based on the [SteamAppPathProvider](https://github.com/Trico-Everfire/SteamAppPathProvider) library by [@Trico Everfire](https://github.com/Trico-Everfire) and [Momentum Mod](https://momentum-mod.org) contributors.
 - `vpkpp`'s GCF parser was contributed by [@bt](https://github.com/caatge) and [@ymgve](https://github.com/ymgve).
-- `vpkpp`'s WAD3 parser was contributed by [@ozxybox](https://github.com/ozxybox).
+- `vpkpp`'s ORE parser is based on [reverse-engineering work](https://github.com/erysdren/narbacular-drop-tools) by [@erysdren](https://github.com/erysdren).
+- `vpkpp`'s WAD3 parser/writer was contributed by [@ozxybox](https://github.com/ozxybox).
 - `vtfpp`'s write support is based on work by [@Trico Everfire](https://github.com/Trico-Everfire).
 
 ## Gallery

include/sourcepp/String.h

@@ -12,6 +12,16 @@ namespace sourcepp::string {
 
 [[nodiscard]] bool contains(std::string_view s, char c);
 
+/// A very basic regex-like pattern checker for ASCII strings. Supports:
+///   %? - matches any character
+///   %w - matches a whitespace character (defined by std::isspace)
+///   %a - matches a letter (A-Z, a-z)
+///   %u - matches an uppercase letter (A-Z)
+///   %l - matches a lowercase letter (a-z)
+///   %d - matches a single digit (0-9)
+///   %% - escaped percent character
+[[nodiscard]] bool matches(std::string_view in, std::string_view search);
+
 [[nodiscard]] bool iequals(std::string_view s1, std::string_view s2);
 
 void ltrim(std::string& s);

include/vpkpp/PackFileType.h

@@ -8,6 +8,7 @@ enum class PackFileType {
 	FPX,
 	GCF,
 	GMA,
+	ORE,
 	PAK,
 	PCK,
 	VPK,

include/vpkpp/format/ORE.h

@@ -0,0 +1,30 @@
+#pragma once
+
+#include <sourcepp/parser/Binary.h>
+
+#include "../PackFile.h"
+
+namespace vpkpp {
+
+constexpr std::string_view ORE_EXTENSION = ".ore";
+
+class ORE : public PackFileReadOnly {
+public:
+	/// Create an ORE file
+	static std::unique_ptr<PackFile> create(const std::string& path);
+
+	/// Open an ORE file
+	[[nodiscard]] static std::unique_ptr<PackFile> open(const std::string& path, const EntryCallback& callback = nullptr);
+
+	[[nodiscard]] std::optional<std::vector<std::byte>> readEntry(const std::string& path_) const override;
+
+	[[nodiscard]] Attribute getSupportedEntryAttributes() const override;
+
+protected:
+	explicit ORE(const std::string& fullFilePath_);
+
+private:
+	VPKPP_REGISTER_PACKFILE_OPEN(ORE_EXTENSION, &ORE::open);
+};
+
+} // namespace vpkpp

lang/c/include/vpkppc/PackFileType.h

@@ -10,6 +10,7 @@ typedef enum {
 	VPKPP_PACK_FILE_TYPE_FPX,
 	VPKPP_PACK_FILE_TYPE_GCF,
 	VPKPP_PACK_FILE_TYPE_GMA,
+	VPKPP_PACK_FILE_TYPE_ORE,
 	VPKPP_PACK_FILE_TYPE_PAK,
 	VPKPP_PACK_FILE_TYPE_PCK,
 	VPKPP_PACK_FILE_TYPE_VPK,

lang/c/include/vpkppc/format/ORE.h

@@ -0,0 +1,9 @@
+#pragma once
+
+#include "../PackFile.h"
+
+// REQUIRES MANUAL FREE: vpkpp_close
+SOURCEPP_API vpkpp_pack_file_handle_t vpkpp_ore_create(const char* path);
+
+// REQUIRES MANUAL FREE: vpkpp_close
+SOURCEPP_API vpkpp_pack_file_handle_t vpkpp_ore_open(const char* path, vpkpp_entry_callback_t callback);

lang/c/src/vpkppc/_vpkppc.cmake

@@ -4,6 +4,7 @@ add_pretty_parser(vpkpp C
         "${CMAKE_CURRENT_SOURCE_DIR}/lang/c/include/vpkppc/format/FPX.h"
         "${CMAKE_CURRENT_SOURCE_DIR}/lang/c/include/vpkppc/format/GCF.h"
         "${CMAKE_CURRENT_SOURCE_DIR}/lang/c/include/vpkppc/format/GMA.h"
+        "${CMAKE_CURRENT_SOURCE_DIR}/lang/c/include/vpkppc/format/ORE.h"
         "${CMAKE_CURRENT_SOURCE_DIR}/lang/c/include/vpkppc/format/PAK.h"
         "${CMAKE_CURRENT_SOURCE_DIR}/lang/c/include/vpkppc/format/PCK.h"
         "${CMAKE_CURRENT_SOURCE_DIR}/lang/c/include/vpkppc/format/VPK.h"
@@ -22,6 +23,7 @@ add_pretty_parser(vpkpp C
         "${CMAKE_CURRENT_LIST_DIR}/format/FPX.cpp"
         "${CMAKE_CURRENT_LIST_DIR}/format/GCF.cpp"
         "${CMAKE_CURRENT_LIST_DIR}/format/GMA.cpp"
+        "${CMAKE_CURRENT_LIST_DIR}/format/ORE.cpp"
         "${CMAKE_CURRENT_LIST_DIR}/format/PAK.cpp"
         "${CMAKE_CURRENT_LIST_DIR}/format/PCK.cpp"
         "${CMAKE_CURRENT_LIST_DIR}/format/VPK.cpp"

lang/c/src/vpkppc/format/ORE.cpp

@@ -0,0 +1,29 @@
+#include <vpkppc/format/PAK.h>
+
+#include <vpkpp/format/ORE.h>
+
+#include <sourceppc/Helpers.h>
+
+using namespace vpkpp;
+
+SOURCEPP_API vpkpp_pack_file_handle_t vpkpp_ore_create(const char* path) {
+	SOURCEPP_EARLY_RETURN_VAL(path, nullptr);
+
+	auto packFile = ORE::create(path);
+	if (!packFile) {
+		return nullptr;
+	}
+	return packFile.release();
+}
+
+SOURCEPP_API vpkpp_pack_file_handle_t vpkpp_ore_open(const char* path, vpkpp_entry_callback_t callback) {
+	SOURCEPP_EARLY_RETURN_VAL(path, nullptr);
+
+	auto packFile = ORE::open(path, callback ? [callback](const std::string& path, const Entry& entry) {
+		callback(path.c_str(), const_cast<Entry*>(&entry));
+	} : static_cast<PackFile::EntryCallback>(nullptr));
+	if (!packFile) {
+		return nullptr;
+	}
+	return packFile.release();
+}

lang/csharp/src/vpkpp/Format/ORE.cs

@@ -0,0 +1,52 @@
+using System;
+using System.Runtime.InteropServices;
+
+namespace vpkpp.Format
+{
+    using EntryCallback = Action<string, Entry>;
+
+    internal static unsafe partial class Extern
+    {
+        [DllImport("vpkppc")]
+        public static extern void* vpkpp_ore_create([MarshalAs(UnmanagedType.LPStr)] string path);
+
+        [DllImport("vpkppc")]
+        public static extern void* vpkpp_ore_open([MarshalAs(UnmanagedType.LPStr)] string path, IntPtr callback);
+    }
+
+    public class ORE : PackFile
+    {
+        private protected unsafe ORE(void* handle) : base(handle) {}
+
+        public static ORE? Create(string path)
+        {
+            unsafe
+            {
+                var handle = Extern.vpkpp_ore_create(path);
+                return handle == null ? null : new ORE(handle);
+            }
+        }
+
+        public new static ORE? Open(string path)
+        {
+            unsafe
+            {
+                var handle = Extern.vpkpp_ore_open(path, 0);
+                return handle == null ? null : new ORE(handle);
+            }
+        }
+
+        public new static ORE? Open(string path, EntryCallback callback)
+        {
+            unsafe
+            {
+                EntryCallbackNative callbackNative = (path, entry) =>
+                {
+                    callback(path, new Entry(entry, true));
+                };
+                var handle = Extern.vpkpp_ore_open(path, Marshal.GetFunctionPointerForDelegate(callbackNative));
+                return handle == null ? null : new ORE(handle);
+            }
+        }
+    }
+}

lang/csharp/src/vpkpp/PackFileType.cs

@@ -7,6 +7,7 @@ public enum PackFileType
         FPX,
         GCF,
         GMA,
+        ORE,
         PAK,
         PCK,
         VPK,

src/sourcepp/String.cpp

@@ -21,6 +21,43 @@ bool string::contains(std::string_view s, char c) {
 	return std::find(s.begin(), s.end(), c) != s.end();
 }
 
+bool string::matches(std::string_view in, std::string_view search) {
+	int inPos = 0, searchPos = 0;
+	for ( ; inPos < in.length() && searchPos < search.length(); inPos++, searchPos++) {
+		if (search[searchPos] == '%') {
+			if (++searchPos == search.length()) {
+				return false;
+			}
+			switch (search[searchPos]) {
+				default:
+				case '?': // wildcard
+					break;
+				case 'w': // whitespace
+					if (!std::isspace(in[inPos])) return false;
+					break;
+				case 'a': // letter
+					if (!(in[inPos] >= 'a' && in[inPos] <= 'z' || in[inPos] >= 'A' && in[inPos] <= 'Z')) return false;
+					break;
+				case 'u': // uppercase letter
+					if (!(in[inPos] >= 'A' && in[inPos] <= 'Z')) return false;
+					break;
+				case 'l': // lowercase letter
+					if (!(in[inPos] >= 'a' && in[inPos] <= 'z')) return false;
+					break;
+				case 'd': // digit
+					if (!std::isdigit(in[inPos])) return false;
+					break;
+				case '%': // escaped percent
+					if (in[inPos] != '%') return false;
+					break;
+			}
+		} else if (in[inPos] != search[searchPos]) {
+			return false;
+		}
+	}
+	return inPos == in.length() && searchPos == search.length();
+}
+
 bool string::iequals(std::string_view s1, std::string_view s2) {
 	return std::ranges::equal(s1, s2, [](char a, char b) { return std::tolower(a) == std::tolower(b); });
 }

src/vpkpp/PackFile.cpp

@@ -18,6 +18,7 @@
 #include <vpkpp/format/FPX.h>
 #include <vpkpp/format/GCF.h>
 #include <vpkpp/format/GMA.h>
+#include <vpkpp/format/ORE.h>
 #include <vpkpp/format/PAK.h>
 #include <vpkpp/format/PCK.h>
 #include <vpkpp/format/VPK.h>

src/vpkpp/_vpkpp.cmake

@@ -6,6 +6,7 @@ add_pretty_parser(vpkpp
         "${CMAKE_CURRENT_SOURCE_DIR}/include/vpkpp/format/FPX.h"
         "${CMAKE_CURRENT_SOURCE_DIR}/include/vpkpp/format/GCF.h"
         "${CMAKE_CURRENT_SOURCE_DIR}/include/vpkpp/format/GMA.h"
+        "${CMAKE_CURRENT_SOURCE_DIR}/include/vpkpp/format/ORE.h"
         "${CMAKE_CURRENT_SOURCE_DIR}/include/vpkpp/format/PAK.h"
         "${CMAKE_CURRENT_SOURCE_DIR}/include/vpkpp/format/PCK.h"
         "${CMAKE_CURRENT_SOURCE_DIR}/include/vpkpp/format/VPK.h"
@@ -23,6 +24,7 @@ add_pretty_parser(vpkpp
         "${CMAKE_CURRENT_LIST_DIR}/format/FPX.cpp"
         "${CMAKE_CURRENT_LIST_DIR}/format/GCF.cpp"
         "${CMAKE_CURRENT_LIST_DIR}/format/GMA.cpp"
+        "${CMAKE_CURRENT_LIST_DIR}/format/ORE.cpp"
         "${CMAKE_CURRENT_LIST_DIR}/format/PAK.cpp"
         "${CMAKE_CURRENT_LIST_DIR}/format/PCK.cpp"
         "${CMAKE_CURRENT_LIST_DIR}/format/VPK.cpp"

src/vpkpp/format/FPX.cpp

@@ -2,7 +2,9 @@
 
 #include <filesystem>
 #include <FileStream.h>
+#include <sourcepp/String.h>
 
+using namespace sourcepp;
 using namespace vpkpp;
 
 FPX::FPX(const std::string& fullFilePath_)
@@ -26,14 +28,21 @@ std::unique_ptr<PackFile> FPX::create(const std::string& path) {
 }
 
 std::unique_ptr<PackFile> FPX::open(const std::string& path, const EntryCallback& callback) {
-	auto fpx = FPX::openInternal(path, callback);
-	if (!fpx && path.length() > 8) {
-		// If it just tried to load a numbered archive, let's try to load the directory FPX
-		if (const auto dirPath = path.substr(0, path.length() - 8) + FPX_DIR_SUFFIX.data() + std::filesystem::path{path}.extension().string(); std::filesystem::exists(dirPath)) {
+	std::unique_ptr<PackFile> fpx;
+
+	// Try loading the directory FPX first if this is a numbered archive and the dir exists
+	if (path.length() >= 8) {
+		auto dirPath = path.substr(0, path.length() - 8) + "_fdr.fpx";
+		auto pathEnd = path.substr(path.length() - 8, path.length());
+		if (string::matches(pathEnd, "_%d%d%d.fpx") && std::filesystem::exists(dirPath)) {
 			fpx = FPX::openInternal(dirPath, callback);
+			if (fpx) {
+				return fpx;
+			}
 		}
 	}
-	return fpx;
+
+	return FPX::openInternal(path, callback);
 }
 
 std::unique_ptr<PackFile> FPX::openInternal(const std::string& path, const EntryCallback& callback) {