Add Memory Dumps Folder option generates 0 pointers #15
Comments
|
Yeah, please upload your memory dumps. I suspect it might be related to the naming of your memory dumps since they the addresses are drawn from them. |
|
Sure thing. It's too big for Github, so I'll put it on GDrive. I've got them named as GameTitle_0xStartingAddr_0xEndingAddr_0xAddr.bin and it looks like either way the correct addresses get parsed and filled in. Just something about adding them all at once messes up. Significant settings for this set are: https://drive.google.com/open?id=1Jaqe_KPj4G339pz50Hc3KXzpskrfAxrE |
|
It also looks like adding them in different orders results in different pointers being shown, although that's probably just based on the first one added... My TempAR does something similar, so I kinda understand that. But it shouldn't show 0 pointers, no matter what order they're added, so I'm not quite sure what's causing that. |
|
Any ideas about why this would be happening? |
|
I just tried it and it looks normal to me. If I add the entire folder, it gives me 4 memory dumps. When performing a search, I get 0 results. If I delete the 4th memory dump and perform a search, I get 4 results. If I manually add all 4 memory dumps and perform a search I also get 0 results. Where exactly do you see the bug? As you seem to have noticed, the first memory dump is used as the base and the others are only used to filter the results of the first memory dump. All valid pointers will be outputted regardless of the order they were added since they would pass all memory dump checks. You can see that happening if you e.g. add the 1st memory dump, then the 3rd, then the 2nd and perform a search. You get 4 results, like before. Also the same thing for adding the 3rd, 2nd and first. Also note that adding the ending address of the memory dump to the file name is redundant for my software since it will read the file size of the memory dump and therefore does not need to be told where the memory dump ends. The target address is parsed from the right side of the file name (ignoring the extension), that's why it still detected the address(es) correctly. |
|
The ending address is part of the dumping software... It actually names them Starting_Ending_DumpNumber.bin . It's important to me to know what the ending address is because we ONLY dump what is necessary, so I keep it there. It hasn't seemed to affect the pointer search, so I kept it that way. :P I wasn't sure if you specifically programmed it that way or not, and that explanation makes sense. Glad you did it as you did, as that's very helpful. The issue is, that if a pointer is found in dump order 1,3,2,4 then it should also be found in dump order 4,3,2,1... at least, I would think it should be... I've found pointers in all 4 dumps that are valid, and can be followed exactly in HxD. Maybe I need to look better at the fourth dump, but I could have sworn that I had followed the same pointers in it. For instance, this pointer works in all 4 dumps : But, I also just did what you did and took out 1 address from the 4. I wasn't sure which one you took out, so I added the folder, then took out 1 address 4 different times. All resulted in 0 pointers, except when I removed the "...F88.bin" address. Removing that one showed [[0x81776558] + 0x24] + 0x1238 in the pointer list. Following that pointer, in the missing dump found that one of the addresses pointed to was in the 0x89's, outside the dumped range. So that's correct, at least. It explains why THAT pointer doesn't appear if the "...F88.bin" dump is used. So, then I tried switching the remaining dumps around. Dumps 1-2-3 found the above pointer, so does 1-3-2. but dumps 3-2-1 and 3-1-2 found several pointers, including the above one. But 2-1-3 and 2-3-1 found 0 pointers, not even the one above that I've already checked is valid in all 3 dumps. I find it odd that simply changing the order of the pointers can either show 1, many or 0 pointers, even if there ARE pointers that can be followed in every single one. I'm not sure how your pointer searcher decides which pointers to throw out and when, but that seems to be the issue, the more I look into it... Not the adding a folder at a time, but just changing order seems to break some results... In TempAR (the pointer searcher I kinda take care of) the way it searches I think is that it find all pointers to a single dump, then it finds all pointers to the second dump, and compares the lists and any pointers that they share get "Upgraded" to a higher level. In the next dump, it does the same but only compares the current dump to dump 1 and continues to "Upgrade" pointers. Eventually, you're left with a list of pointers that's earned so many levels (with higher leveled pointers matching more dumps, and lower levels matching only 1 dump)... so no matter the order, if a pointer shows up in all dumps, it will always show up.... unfortunately it also means there's a ton of other addresses of various levels that also show up (Hence why I MUCH prefer Universal Pointer Searcher).... at least they have colors associating the different levels.... I'm not sure how yours works, though... so I'm not sure why sometimes it rejects a pointer simply based on dump order, even if that pointer is valid in all of the dumps.... this post has gotten WAY longer than I meant for it to, I'm so sorry! |
|
Hi @Yohoki, it's been a while. |
|
It sure has been a while. I've mostly left the hacking scene for vita, because some new script kiddies have started releasing really unstable codes that can cause serious damage, and are absolutely adamant that this is how the scene should move forward... I'm not going to be a part of that movement. XD But, I do still hack my own games on my free time, so I'll give this one a run and see if it fixes that issue. I should still have my old dumps on an HDD somewhere on my network. I'll check it out this weekend and see if I can figure out how to use it and if it fixes the problem. edit: I just reinstalled the redist but it's showing errors still. |
Exactly, this kind of stuff always happens. Kind of ruins the fun in this "hobby" which is sad. This makes me not want to release any codes or hacks to begin with, people just steal them and abuse my work sooner or later. Anyway...
Yep, this just surprised me for a moment since it works flawlessly on 3 of my laptops but I got you haha nice job, you already found a bug. You need to type EDIT: |
|
lol, I'm so used to running batch files from cmd that it hadn't even occurred to me that the .exe might be the issue. Ok, i've updated (this is mandatory, it looks like?) I'll read that help file and set it up when I can get a chance to sit down and look at it. I should have a couple hours this weekend that I can sit at my desk and try out moving the dump orders around and things and see if I can break it again. XD |
|
Is "--initial-endian little little little" not correct? Edit: nvm. you changed "--initial-address-size" to "--address-size" at some point, and I was copying the video. XD Makes sense, the game probably isn't going to suddenly go 64bit by itself. lol |
|
update: Original: |
|
Just finished running my first successful run and I'm noticing a few things. you asked for feedback, so I'm gonna give you some feedback. Hopefully it's good and (more importantly) useful to you. Showing exclusions:While probably not useful to most users, I REALLY like seeing WHY it's kicked out the pointers. In these 2 cases, it's found a loop and kicked it out. No point following the loop when we can just enter the pointer from further down the line. Makes sense, and I'd have had to checked HxD before I found out that it was looping. RAM Monitoring:Oh, thank jesus. You were doing me a concern there. O.O I'm concerned that the ram may have halted searching for more pointers, though. But I also don't know an easy way to solve this. I'd rather it stop early, than crash to blue screen, so that's better than before, at least. Documentation:The video looks like it's outdated and some of the --help text is not easy to understand. I was able to get it mostly working by watching the video and comparing my string with the --help text, at least... but had I only had one or the other, I don't think it would have worked. It seems like you're not interested in adding a gui for this one, so if you want it to be used, it's got to be clear how to use it. Thankfully, it should be pretty easy to make a gui (or even a prompt system with a bat file) that simply takes in the user's info, and passes it all to the cmd. I'll probably make a bat file for myself, so if you haven't made anything by then, I'll let you know in case you want to use it. Speed:took 1500 seconds to run for me. My RAM got in the way, and I think my Virus Scanner was also doing a scan at the same time, so I probably had a huge throttling issue going on. but, here's how the times went. I do have one concern here. The cycles step took up the most amount of time, and only excluded a very small percent of the pointers. Could this be moved to a later time in the order of events? Say, after validation, but before duplicates removal? I'm not sure what the validations step does, yet. But if it won't break it to run that step slightly slower, it would maybe cut the running time down by a lot if the excluding cycle was performed with only 5000 results, instead of 20 million. Printing pointers:I didn't see where it printed the pointer out at..... but I also ran it with ">Results.txt" in the commandline.... so If it was supposed to print out to that already, that would be why.... it also didn't display the pointers on screen in the cmd window, but again I printed it out to results.txt hoping to also catch the errors and things in the txt too.... So, that one's probably me messing up your code. Overall, I like it. It's a bit clunky right now as I'm learning it, and it could really use some documentation.... but I like it so far, and it should be pretty easy to make something to pass the arguments to it, so that noobs don't bother you with questions for years. :P |
|
Thanks a lot for taking the time to write such detailed feedback, it's highly appreciated!
Yes, it's mandatory.
Exactly, I renamed those 2 switches.
Yeah, the check is: Essentially the target address count needs to be 1 less than the comparison file paths.
Yeah, I think the
What happens here is that you're at 99% RAM but the excluding cycles step goes through the pointer list and copies over the ones without cycles into a new list which duplicates your RAM usage temporarily until it's done. Since you're already maxed out, it will slow down tremendously due to Windows writing out RAM pages to the disk and swapping them back. One suggestion would be to set the RAM threshold to 85% (this can be configured via command line argument). Changing the order of the steps is a good idea, I will think about that.
Normally they are all printed at the end but via command line switch you can specifically write only the pointer results to a file. You don't need to do the piping. |
No problem. Hopeful it's useful. I don't know how to code well, so I gotta rely on people like you making things that work good. :P
I'm not sure I'm understanding this part. I was trying to copy your video, and it looked like the first example showed you adding 2 dumps as "Initial" dumps and the same 2 again as "Comparison" dumps. I assumed this was to avoid the issue I was having with re-ordering dumps, by running the check against multiple types of both. But, again, the video is probably outdated and has maybe changes considerably since uploading. I'll have to play with it more to figure out how to use it well. I did notice, though that the video only showed 1 target address instead of 2, since there were 2 dumps added as initial/comparison. which didn't make sense to me, so I tried adding 1 address per dump. Possibly that's the issue?
Some of it looks a bit confusing if you don't already know how the input should look... Like this for instance: From the video, I saw this should be "1,3". but will "1-3" work? will simply putting "3" work? If I hadn't seen the video and read the text on screen, it wouldn't have been clear to use a comma, and I probably wouldn't have thought to use one at all. An example would clear it up immediately, for those that don't already know what it should look like. I don't think a GUI is necessary, and if it's tiresome to make one (I know that part is boring) then don't worry about it. There's plenty of command line tools out there for other console things. And what you're making isn't necessarily for the beginner. But I also know you'll get people asking "How use this it broke?" if you don't. So it'd probably be best to at least make sure the readme is clear. I'm also pretty passionate about having easy-to-use interfaces.... So if it's not something you're interested in, don't let my feedback here sway you, because I'm going to be overly talkative about this particular part to begin with. XD Plus, I will most likely make a batch file myself for this that steps through each of the arguments and gives a prompt. If I make it, I'll gladly share it with you to do what you want with. :)
I did run the search again shortly after with the throttle set to 80, and it does look like it cuts off search when the throttle limit is reached.... which is unfortunate, because at 80% I had almost 5million less pointers when done scanning. I definitely lose a good amount of starting pointers before the validations and exclusions for that cost of extra speed. How to go about getting those back in? I don't know... maybe I can step through depths and only search 1 level deep at a time, and then increase to only 2 levels, or something like that. But I'm not sure this would even save any ram usage... ah. I'm not that worried about it. It's my old Mobo's 8gb limit that's the real issue here. But changing the order of steps might help reduce the time waiting at the end. The loops checking seems to take the longest amount of time while cleaning up, so saving it until later might be a good thing. You'd have to try it out and see if it does anything good or just makes the rest take longer. I can't see your code, so I can't even guess what all's going on. :P
Ah. Ya, running it again without the pipe displays it on screen. No need to pipe anyway, because the pipe isn't catching any of the error messages or anything anyway. I'll just use the switch, then. lol |
No. You need to give it the start value, a comma and an end value. I added this clarification to the
Initial means "the set of memory dumps to use as the initial memory snapshot", e.g. some systems have more than 1 memory dump which need to be treated as a single input but it only has 1 target address no matter how many memory snapshot files it is divided into. In your case it's only 1 memory snapshot file though. The comparison memory snapshots are all additional memory dumps. In the video it was I assume the MEM80 and MEM90 dump from Super Mario Galaxy. The comparison memory snapshots are the ones you want to compare/validate the pointers against to sort out the pointers which don't work on all memory dumps. Since you asked for better documentation, I wrote a glossary of the terms I'm using in the Another thing you can do is save pointers to a file and then load that again to validate against another memory snapshot if you don't want to repeat the whole search with another comparison memory snapshot added. This way you can keep filtering more efficiently. Also I added a value to disable memory pointer sorting at the end since it can be very slow on like 100 million pointers lol.
Yes, it stops collecting more pointers when the RAM limit is reached. However, I now implemented an improvement by re-ordering the filtering operations and also saving RAM by deleting from the list instead of setting up a new one. This means it will run through more smoothly with a RAM limit of 99%. I just finished a pointer search with 117.941.132 pointers in ~ 569.865 second(s) but my laptop has 32 GB (20 GB were temporarily taken by the pointer searcher). However, I'm wondering why you really need that many results. If you want even more, maybe we can do something to cut down on those huge pointer lists before they happen? It would speed everything else up as well. You may try some of the other command line options like the concrete last pointer offsets. |
Lol. I'm aware of this now, but if I had only the --help, I wouldn't have even thought of adding a comma... I'd have probably tried "1-3" or "3" or a few other things. But a comma wouldn't have been my goto. Adding examples to the --help would help, I bet. If I couldn't figure it out, and I'm used to hacking and pointer searching, a newbie def won't. :P
Ah. Yes, I did noticed one was MEM80 and MEM90. I wasn't aware that Wii had to have them dumped separately, as I don't own a wii. On Vita, we get to choose the range we want to dump, which is usually 81-83, but it can go into the 90s sometimes, but it's usually all one big file... I've had to make cuts sometimes and add 00s in as padding on a few occasions where there's random bits of system mem tucked inside my dumping areas, and the vita crashes if you try to read these protected areas... so having that option to use split files might actually be useful for us as well.... although, it's quite rare that I've had to do that, and I don't recall anyone else mentioning that on vita yet.
Nice. I'll give that a read, but I think I understand... most of the terminology, myself. But, some of it seems unique to your wii hacking, and I won't know about it yet. The better error messages will definitely help, though.
I will give that a try this weekend when I have the free time to sit down and actually do more than one run at a time. :P
Vita's been pretty difficult to crack some of the games, lol. sometimes the pointers start WAY in the end of the dumps, but are usually rooted somewhere in the 81-83 areas. I think it's because of the way the ARM processor works, by having 2 segments for every module, so those 2nd segments can sometimes be put really far away, even though the first segment is early in RAM. So, having those extra 5million pointers may be necessary for vita, especially if they're from later in the RAM. But, then again, the vita ALSO has a weird issue where almost every game just SOMETIMES loads 0x1000000 bytes out of place, and we have no idea why... There's so much going on on the vita that I WISH I knew more about programming it to help explain what's going on, but I'm climbing mount stupid here. I make good codes, and do some really advanced things that only I know how to do.... but when explaining it to people, I can only give my experience, I don't have the book knowledge of the inner workings to explain what's causing these things we see.... And, unfortunately, since the switch came out, most of the technical people have moved to is as the new handheld, so we're a dying console. :( AND THEN some games, have pointers that aren't static addresses, but dynamically rooted at a static offset from the start of it's memory segment.... so the base address changes every time the memory segment loads or shifts location... Vita's a demon.... don't try hacking it... It will eat your soul away. lmao. Either way, I'm glad my feedback pushed you to making it more efficient. Good job. That was a pretty quick fix that will probably help a lot. Thank you a lot for that.
Hmm... do you mean something like stopping the pointer search midway and doing the excluding/validating/etc and then resuming where it left off? I don't know what would make it more efficient here. You're the expert on your program, so I'll have to trust your word if you think it'll make it more ram efficient. Also... Thank you for your hard work. I don't give you the feedback expecting anything to be changed in the code at all. So, seeing you read it, and then immediately work on a patch or think up a new design is greatly appreciated, but not at all expected. |
|
Since I took a while to release this, I wanted to make it absolutely perfect. You inspired me again to improve and fix a couple things. I'm looking forward to your more detailed feedback once you get around to doing more in-depth testing. I don't think any of my terminology is Wii specific, I took many systems into account while designing this. It even works on Windows games since they have modules which need to be dumped individually and they can be loaded by passing the folder of modules as argument. Furthermore, it has been tested and working for the 3DS, Switch, Vita (as you know), the Nintendo Wii and Wii U at least. Essentially the tool's flexibility supports every system in existance using that terminology. One of the reasons I didn't want to make a GUI is because it gets really complex to support all use cases cleanly with a GUI. Usually a GUI allows less flexibility than using the command line interface as well. |
|
Ya, I wouldn't do a GUI, either, I don't think. Working with TempAR, it's original creator made it have a GUI, but that limited it to only comparing 2 dumps at once, which works but there's no reason why you can't compare more than 2 dumps to make it better. I DID add that and it was a pain in the butt (plus my code is sloppy as hell since I didn't know C# at all) and the GUI was looking pretty ugly for a while. But even now that I've added up to 6 dumps, there's no reason why it COULDN'T compare more than 6, and people have asked for it.... but there's no room left on the GUI, even after I made it larger... So, I'm not adding it. XD |
|
Well, in that case you're doing it wrong, you should use something more scalable like a table/list similar to how I did it in the Java Universal Pointer Searcher GUI and not duplicating the items/fields for each additional memory dump. This way you can allow people to add unlimited rows. However, managing this with all options and possibilities I'm providing in the command line tool gets out of hand quickly for GUI construction and it's so much unnecessary work when you can already use the pointer searcher like this, it's only less user friendly. Normally, I'm also a huge fan of maximum usability but command line tools can have their charm as well. This tool isn't intended for the masses of noobs and kids anyway. Without those guys a tool cannot become super popular which is okay for this one, only active game hackers will be interested in this and they should have a few more brain cells to put to work than the noobs and kids who can't work with the command line. |
|
Well, tempAR was a program I only took over from the PSP days. So, I just edited what was already there... Didn't even have the source code, I had to use DotPeek to decompile it. So, adding more dumps meant cut+Pasting bits of code and hooking them up to boxes in the GUI. I would have loved to make a list like in UPS, but the way it displays pointers is a bit flawed to begin with... Adding more dumps just increases the number of pointers, and it never goes through and erases the ones that don't work in later dumps... It needs a rework, but I'm thinking that'd take a complete rebase, and I've only just made my first C# program from scratch a couple days ago. So, I've just added a few extra features to make vita hacking easier, and left it as is with the source available. Insanely fun project that I'm proud of, but it does have it's flaws. I do agree with the charm of CMD apps. Most of my work is batch-based. :P I still try to make my programs noob friendly, because I make several for my wife who is def a noob with computers, |
Well, yeah, it's quite a mess then and would need a rewrite. Working with just |
I've noticed that the option to load an entire folder of dumps results in 0 pointers being shown. Changing none of the other settings, but removing all addresses and adding them individually fixes the issue. I can upload a zip with the dumps if you need.
The text was updated successfully, but these errors were encountered: