DEFF
REV - REViSi0N => Past, Current And Ongoing Projects => Abandoned Project: WAC 5.0 => Topic started by: Prozedurmeister on 20161211, 15:06:59
-
does not matter where SH3 is installed... I only chose two ships...
That is wrong.
Folders "Programs" and "Programs X86" are under special UAC-protection, remember?
Grey has asked you: Is your UAC active?
And how do you select two ships only?
We have tested by selecting folder "crew" and it worked.
I have fixed that problem with spaces in the name.
Blitz has fixed the bug that wrote only Byte 7.
Have you reindexed the files before using? That means after you ran our tool?
-
If we modify all duplicate IDs using Blitz external software,
the index will not be updated. Is this a problem?
I dont know. Because it is not S3D it has to be tested what will be with the index.
Otherwise I can imagine that the index could be renewed with s3d.
Is this not already the solution?
Now we have a result of this test that you wanted, Han Solo!
Then what is the logic consequence of this? The index must be updated!
This image by PRM shows it:
(https://abload.de/img/s3dids2da15.jpg)
-
Ahoy mates!
That is no problem of the index my friends.
In the picture I uploaded all the duplicated IDs have been changed by your tool.. but the got no new ID.
So.. every node that was changed has no ID after running the duplicate ID Killer.
Every controller that is listed with that exclamation mark needs an ID.
@PT
I know that this is almost not doable --- BUT I did almost for the sea folder right now. And as we know new ships cause instabilities it has the first priority.
Alpha 8 is online and should be used for testing and error fixes. Alph9 should have ID fixes included!
@PM
UAC is active... I could deactivate, but first I need a tool that is working without bugs.
I can copy 2 ships in a seperate folder on my desktop and let the DF do his work. Then there is the correct path for your tool in the clipboard.
Reindexing is not possible with all these nodes without ID.
-
That is no problem of the index my friends.
No? What else do you believe is the problem then?
I had just quoted what you wrote last Werk...
I just wrote that all node that have been changed by the Duplicate ID Killer got no new ID and that no ID is not possible in SH3 files.
At least the files from PT have these errors!
EDIT: Ok... after deactivating UAC your tool is the way working you described but the Problem I described with the files from PT remains!
I just let your tool do his job for only 2 ships and the result was that every "old problem ID" only just got deleted. Your tool does not renew the IDs!
-
Han solo, not everybody is a modder and familiar with S3D. You cannot expect that of everybody - and still we all have our place in the team and contribute our part.
Look at what and how much Blitzkrieg and Prozedurmaster do to help you!
And if you insist on doing so much manually, why not invite modders like makman into the team?
Just my 2 cents - from somebody who has never tried to mod and still tries to be a good dog ;)
That is true. I can not expect that everyone is familiar with S3d as I wrote above.
But as a teacher I can say that the nescessary knowledge for the fix of IDs can be done by a 9 year old child.
I appreciate your efforts in coding the programme but it can not be expected to do all the ID fixes manually by only 2 Team members if your programme does not work.
That would break the way teamwork is meant to be.
Makman does not want to enter TBT, he has his reasons which I do understand.
@Prozedumeister
Main chunk ID is the node that I surrounded with a red circle in the upper legt corner.
(https://abload.de/img/4weeskcxp1jve.jpg)
You See, is has no ID 'N/A' (not available) because of the Duplicate ID Killer.
-
I see what you mean...
It looks like if we change the ID within S3D, it automatically updates the index,
but when we do it through Blitz software, the index is not updated and then S3D gets confused.
It's looking the index for ID X and it's finding ID Y instead. Soooo... the index must be updated as well somehow. I don't even know if this is doable.
This is exactly what I wrote in http://www.donitzeliteflotilla.com/forum/index.php?msg=822
So the questions are:
- Where is the index stored? Do the Parent IDs need be changed? If yes, how?
- How can the index be accessed? In what format and where is that index?
- what needs to be done that s3d actually sees the new IDs and displays them?
-
That is precisely why He demanded such an example in this post:
http://www.donitzeliteflotilla.com/forum/index.php?msg=684
And if I do not read the logfile wrong, this is software in ALPHA state - I.e. for being tested,
so isnt that self-evident? And he did say BACKUPS FIRST, too !!
So I have no idea what needs to be fixed because of the tool. Don't you make backups?
Again: can my questions please be considered, too?
I was basically stating the following: If I ask Honda to design a new engine for my racing car,
and when they deliver the engine it doesn't start at all, then there was no point, was there?
They can say "but hey, you asked for the engine and we delivered, here it is, it's beautiful and chrome and has lots of valves."
The bottomline is, the engine doesn't start. It's paperweight.
Now, regarding the software and why it's not working.
I'll try to explain this as best as I can, with my understanding of the problem.
The file contains an ID. This ID is linked to many other things and sometimes even have children IDs.
When you change this ID in S3D, it randomly generates another ID to replaced it, and automatically updates
all the indexes and children IDs so they all work properly with the new ID.
It's like if we replaced Blitz with another guy from China, but never told anybody else in the team. You don't know what's going on.
All of the sudden Blitz is talking Chinese to you and you don't know why.
By changing the IDs using Blitz software, everything else in the dat/dsd/sim/zon files are not being informed of this change, index is not being updated.
By changing the IDs using S3D software, everything else in the dat/dsd/sim/zon files are being informed of this change, index is updated. Therefore it works.
Now, I'm in no way trying to say Blitz and Proz codded a bad piece of software.
For all intents and purposes it does what Blitz and Proz wanted the software to do.
It just doesn't solve our problem. Maybe I should have explained this better before.
But then again, we only discovered the index problem AFTER trying to use Blitz/Proz tool.
I'm thinking that maybe this is a bit too complex to code in such a short time.
But maybe I'm wrong, I'm no codder.
-
I also respect what this neat team achieves.
Blitz has really brought people together to get something done.
At least I would not be here without the Blitzkrieg coming over me 😂
And good explanation, but we have been there already, so lets think forward:
So the questions are:
- Where is the index stored? Do the Parent IDs need be changed? If yes, how?
- How can the index be accessed? In what format and where is that index?
- what needs to be done that s3d actually sees the new IDs and displays them?
-
- Where is the index stored? Do the Parent IDs need be changed? If yes, how?
- How can the index be accessed? In what format and where is that index?
- what needs to be done that s3d actually sees the new IDs and displays them?
[/quote]
As I never bothered to load one of these files into an HEX Editor, all my answers are strictly speculative:
1) Must be inside the same file. Each file has it's own index.
2) No idea.
3) No idea.
Hence why I said maybe you guys cannot figure this out in time.
At first we thought it could be something simply achievable.
As of recently we've learned things are far more complex.
To be honest, in order to craft such a software that we need for this task,
I'd think that a good sit-down-chat with Skwas, the programmer behind S3D, would be required.
http://s3d.skwas.com/about.aspx
If there is a way to contact this guy and if he's willing to shine a light on this issue
and give us a hand, then probably coding this piece of software could become childplay.
His email is skwas@skwas.net. Maybe someone with a bigger "name" such as Hansolo could give it a go? =)
-
i like to see this heavy here activity while Prozedurmeister and I are trying to find new ways.
His email is skwas@skwas.net. Maybe someone with a bigger "name" such as Hansolo could give it a go? =)
Same problem as with all modders on Subsim: I talked to Skwasjer in summer (or last year) and have not read or heard any lifesine ever since. Same with Dion/NYGM, Canovaro (x-TBT), Neal Stevens........ really most of all over there.
So yes, I did give that a try, but no, no response - since months.
All we can do is take Shark's input into consideration. Gunnar? What do you think about Shark's questions? Do you know more?
-
Discussing only does not bring us closer to a reasonable solution!
Obviously it is not sufficient to only overwrite the existing Duplicate-Node-IDs,
because S3D apparently changes other things too: Shark is good when writing about the index!!!
Because BK and I are not so good at practicing with S3D, we therefore need more detailed information:
- Where and how precisely do you guys change the IDs -> all steps needed
- We need ALL the resulting files of at least two units for a comparison to see what S3D changes in ALL files per unit
I therefore kindly ask you to submit us example files for at least two units (all files per unit)
so that we can try to find what the nodes and changes mean and how S3D does all this.
Only then we can extend our working program that now writes the IDs errorfree (bugs fixed)
but also rewrites the connected nodes and the index
Let me help you, Proz.
1) Download S3D if you don't have it.
2) Open S3D, open ANY .dat/dsd/sim/zon file in S3D
3) Expand ANY node, find the ID line (on the top right)
4) Click "NEW" besides the ID, S3D will generate another ID for it.
5) Save-as a new file, so you have both the old and the new to compare.
Now you can explore both files and check all the differences using whatever tools you prefer, like hex editor.
Here's the location of the ID and "new":
(https://s23.postimg.cc/ggjpesjrv/Capture.jpg)
-
Blitz and I are sitting at this problem and we found out that the Dup-Finder only lists the MAIN CHUNK IDs, nothing else. He simply skips all other occurances of duplicates and does not mention them!
=> This is the reason why our Dup-Killer could not work with the results of the Dup-Finder!
We have today finished new code that finds also all the other IDs that are connected to the Main Chunk ID that Dup-Finder lists in its results.
This is the output of our new code
found: 0CD4420B8480A380 at offset #00000032/$00000020
found: 0CD4420B8480A380 at offset #00827263/$000C9F7F <- this is the only one found by the Dup-Finder!!!
found: 0CD4420B8480A380 at offset #00827292/$000C9F9C
found: 0CD4420B8480A380 at offset #00827404/$000CA00C
found: 0CD4420B8480A380 at offset #00827497/$000CA069
found: 0CD4420B8480A380 at offset #01664188/$001964BC
We are now discussing how to write all additional findings
Greetings from Blitz and me!
-
Yes. Correct.
Now we need to get feedback from you Gunnar and you, Philipp.
Please take a look at the log file and tell us if these functions are enough for you.
-
... can you upload the programme?
Not before you have commented the logfile
A logfile is theory... from my point of view it should work. But I am no coder and do not have knowledge of SH3 file code.
So.. if we could test ist, we would know more!
EDIT: Accomplished my mission of fixing duplicated IDs for the sea folder !! 8) :o ??? :D ;D DUplicated IDs->Sea folder -> zero/0/nothing !!!!
-
A logfile is theory???
Have you taken a look at the logfile yes or no?
To me it looks as if the new version shows pretty well what it does and does what you wanted - as far as that can be guessed. So please say clearly what you want or need.
@han
Note that you make colaboration really difficult when you never answer clearly when you are asked something. Same with my comments - you did not react on ANY of my 4 posts!
----- removed 1 dead link -----
----- removed 1 dead link -----
----- removed 1 dead link -----
----- removed 1 dead link -----
@Blitz and all others:
I thought my posts were no useless trash and could help the matter , but
If I am totally ignored by HAN here, I will not bother about helping with WAC anymore.
What I am reluctant to do is waste my time on niggerwork that a Blitz-tool is going to do anyway - one week earlier or later does not matter...
-
While working through the whole thread I am now summarizing the problem.
Therefore let me verify/falsify a couple of assumptions and hypotheses:
Hans, what exactly is this "Remap IDs" under "Tools" in S3D? Wouldn't that help us?
If you use the remap Option a whole unit(file) will geht everywhere New IDs.
So in that case we would have to reassign everything again. Shipwaves, damage files, sounds, etc.
Remap only recreates the IDs _within_ that one selected unit you just opened.
But it is unclear what you mean by "have to reassign everything".
PM just confirmed that there are no other references that need to be reassigned - from where have you got that, Gunnar?
We are JUST NOW talking about this online - so if you could come online and talk this would really bring us forward and clear another misunderstanding.
Quote from: Grey Eminence on 20161212, 20:34:51
What pisses me off is that it seems we are trying to fix years of lazyass work from other modders who
couldn't be bothered to check if there were duplicated IDs for the ships they were creating copying others.
True! And what pisses me off even more is that we only have this assumption and no evidence that the way proposed WILL work.
I admire Blitz for taking all the efforts!!
Grey is correct. Read on why.
Quote from: Philipp_Thomsen on 20161122, 01:04:29
Reason for this is, for some reason some of these IDs are duplicated, so when the game calls for an ID already in use, game crashes.
By replacing these IDs with new IDs, we should be preventing these crashes.
This hypothesis is most apparently wrong. Read on as to why.
Quote from: HanSolo78 on 20161119, 09:22:14
The origin of these duplicated IDs is probably Bad cloning from other units. This tool appears the First time on subsim and I think not many do know it. So Most people were not aware that the new units they did have duplicated IDs with existing ones. It was always in my mind that this could be very complicated ... new units... No one ever had a look to the entire mega mods.
This assumption is partly wrong.
The "problem" of duplicated IDs (if there is one, read on!) has not come to life with the mods or mega-mods!
Reason: I have scanned GWX3 and WAC4 with the D-Finder and they both have duplicated IDs, too!
Even stock 1.4b has! And all them three do not crash.
Conclusion: The crashes do NOT have their root in the duplicated IDs, otherwise GWX3, WAC4 and stock1.4 would crash, too (which they do not)
Quote from: HanSolo78 on 20161122, 14:21:57
What is the root cause of these duplicated IDs?
Who cloned what and where?
Does WAC 4.1 also have these duplicated IDs?
Do also other mods such as GWX have these duplicated IDs?
I sense there might be a totally different problem that came much later than WAC 4.1...
The root is every ship/unit creator. Every unit is cloned from an existing unit. Then the IDs are renewed by the creators. Unfortunately only Most of IDs from the .dat files were only renewed, not all. IDs from all other ship files are more often duplicated.
WAC4.1 has also duplicated IDs, but probably not that much. Gwx I dont know. I dont have it installed. But it does not change one Thing. The problem is real and there is nothing that can be done except solving this problem.
As explained above, this problem is no problem for WAC4, GWX3 and stock.
So it cannot be the IDs that cause WAC5 to crash.
Prozedurmeister was correct when doubting it - he was also asking for proof:
duplicated IDs which probably cause the CTDs as a unit which has duplicated IDs also cause SH3 to collapse.
Is that certain?
Most obviously, following the above argumentation, this is not the case.
It is of course true that Duplicated IDs are no good, and certainly even PACK3D only copied existing IDs.
But I fail to see why GWX3 and WAC4 would not crash - they contain bunches of these duplicated IDs as well!
-
Then... another question from me. Why so new ships lead to CTDs. New ships contain new different IDs, which are not in Gwx3, WAC4.1 or Stock SH3. There are totally different ships and IDs.
It is easy to create a new theory.
It is definitily prooved that duplicated IDs lead to CTDs.
So... any news theory needs proof.
What about the proof of your new theory? What else could be responsible?
Btw. Happy New Year
-
Who has proved that, Gunnar?
-
It is easy to create a new theory.
What about the proof of your new theory? What else could be responsible?
Which new theory?
I am not creating a new theory at all.
I am critically reflecting what is being assumed here.
Then... another question from me. Why so new ships lead to CTDs.
Try as I did and run the Duplicate-Finder on WAC4.1
WAC4 was NEVER reported buggy or crashing. Or was it?
And you will see that WAC4 has loads of duplicate IDs, too. But it does not crash!
So what does that tell you? Judge yourself.
It is definitily prooved that duplicated IDs lead to CTDs.
Then please explain why WAC4 does NOT crash - though it has loads of duplicated IDs.
Also please do not forget to answer all the other open questions - shall I mark them with color?
I fear that you overread them...
-
@PM
Me, I myself prooved that a duplicated ID in a File lead to CTDs.
@ Blitz
There is only one sentence marked as a question.
And maybe it ist mit clear what I meant with reassign. It means that all the other files like! .val, .sim and so on need a new connection to the dat file. Because the whole dat file gets new IDs with the remap operation.
-
Standing in the shoulders of giants, we can reach higher. That's the way to progress
...and so we did...
BACKUP FIRST!
EDiT:
Moved all these instructions to http://www.donitzeliteflotilla.com/forum/index.php?topic=2660
BUT!
Please do not blame us - we have not gotten confirmation that what we coded is wanted!
Reason: Gunnar, you have written posts but have not responded to these more important messages:Re: Problem: Duplicated Ids (Nodes)
(http://www.donitzeliteflotilla.com/forum/index.php?topic=142.msg877#msg877)Re: Problem: Duplicated Ids (Nodes) (http://www.donitzeliteflotilla.com/forum/index.php?topic=142.msg885#msg885)
Re: Problem: Duplicated Ids (Nodes) (http://www.donitzeliteflotilla.com/forum/index.php?topic=142.msg891#msg891)
Okay, so this is a screenshot of our newest version - both tools in one
(and checked only as far as we can know without our questions being answered!)
(http://abload.de/img/ss_dk10xvk0q.png)
-
@PM
Me, I myself prooved that a duplicated ID in a File lead to CTDs.
And how?
Have you read what Blitz wrote?
"Then please explain why WAC4 does NOT crash - though it has loads of duplicated IDs."
If what you say is true then why do WAC4, GWX3 and stock not crash - only WAC5alpha crashes!
And in what environment and what circumstances and after which action do these crashes come?
Why do you never answer all that?
Please explain that contradiction!
-
Here is the breakdown of how to change these manually in case someone wants to help.
1) Run the duplicate ID finder in a specific folder, like crew, sea, objetcs, etc.
2) Copy all the results into google spreadsheets or excel if you have it
3) Open the ID reverser Blitz and Proz made together with the excel list and S3D in the same window, as shown below:
(https://s24.postimg.cc/i0b2rwjkl/Untitled.jpg)
1) First line contain the ID in question, ignore the second line and grab the name of the file that needs fixing from the third line. Open that file in S3D.
2) Copy the ID for that file, toss it in the converter and convert it
3) Copy the result of the conversion into the find function of S3D
4) Hit "NEW" to generate a new ID for that node, and hit CTRL+S to save.
5) Move on to the next file.
6) Profit.
-
Sorry, guys. Prozedurmeister and I were so discouraged by the circumstance that our questions remain STILL UNANSWERED,
that we ourselves then neither bothered about testing with the uploaded version.
So yes, we did test within our compiler-environment, and there it worked
but no, the download-version had "WRITE= OFF".
This is now the WRITE-enabled version for all of you to use. We have compared the files in hexeditor WITH THIS VERSION
and the changes are good.
-
Ok, the tool apparently works, the IDs are being replaced and opening the new file in S3D doesn't show any error.
However, we have a new problem. The duplicated IDs are being all replaced by a new ID and so this new ID is also a duplicate.
See, the Duplicate finder was run again and a few duplicate IDs were still found.
A lot less duplicates than before... which is weird. I'm not quite sure why.
PT, I coincidently found that you provided a screenshot - I hate that postimage does often not work properly, so I converted it to another pic-hoster:
(https://abload.de/img/untitledfdol1.jpg)
This screenshot is a perfect example on how frustrating our work is.
A) Weeks ago we did not know nothing about how IDs were written, the format, the headers, the index.
We reverse-engineered all that - stumbling in the dark.
And we still to not know the format of the index, thus we cannot even know if we are doing good.
B) The output of Ref's Duplicate Finder is total crap - which of each duplicate pairs should we work with?
If we took path#2 (second one as duplicate), we would in larger lists only write to the same file again and again - so we fixed that.
If we took path#1 (we thought of that as unlogic), we would in this case only fix 964121 and 964261 - and 177487 would be skipped !!!
Following this unlogic nonsense I decided that we write a routine that searches the IDs WITHIN the file of path#1 and collect these offsets to fix them ALL.
C) What we here do not get is why a specific range of files has apparently the problem that you describe, this happens with...
EXACTLY 15 files in the AIR folder
EXACTLY 1 file in the CREW folder
and so on.
what is so special about these files?
Even if they can't, this already helps a lot.
If the new ids generated indeed work I'm sh3 (haven't tested) then the tool does 90% of the work, we can polish the rest by hand.
I dearly hope so. Losing too much hair on this...
-
We are trying to find why certain units "behave" different.
It would appear this is because the images are SUB-EMBEDDED so to say. They contain a sub-link and an EMPTY (!) node-description !!!
Look at this screenshit:
(https://abload.de/img/emsubpougk.png)
-
To Oliver and Peter... I just worked with your updated tool and it definitely cna save us much time!! :-* :)
Great work from both of you! I just reworked the submarine folder from about 28000 line to about 2900 with your tool! :)
Many thanks for this final acknowledgement.
Hearing this from you as a teacher really made my heart rejoice :D
There are some things that can be done ;)
-Manual ?
-Add on Mods (different GUIs, harder AI)
-What else? Your ideas?
That is all here on this forum already!
I collected everything from our emails that is about features in this thread
Now, lets pack this together and start testing Alpha 9 for CTDs.
If it's all running smoothly, then lets start talking about modding it further.
Good point!
-
Maybe I didnt understand the problem.
If I remap ID using S3editor (v0.9.9), the parent ID in .cam file remain linked to .dat ID.
So I think it is not necessary to reassign.
I tried and the result is as I told above.
Let me know if I'm wrong.
I'm glad to help you.
-
You have not to remap a single id.
Open all files (dat, cam ......) and keep them open
Now in .dat click 'tools', 'remap id', 'random', 'ok' .
Then in each orher file (cam, sim, val.....) click 'tools', 'remap id', 'ok' (DO NOT CLICK RANDOM. Click it only in dat file).
When done this, check if the new files have no duplicate id. If so repeat the procedure.
I use this metod because there is no possibility to forget any parent ID ( i think this has been the cause of missed parent id in cam).
I will send you an interior remapped so you will see :)
-
Sorry....
I just opened the interiors files and I saw that there are duplicate id in the same file. Do not consider my precedent post.
I will remap the interior.
I need the following files (error while processing file - cant open with S3ditor)
- nss_U-Boot2_rr.cam
- nss_U-Boot7_rr.cam
- nss_U-Boot9_rr.cam
21_rr works :)
-
HanSolo, but you can open those files?
S3ditor return me an error. I cant open.
Can someone try to open them?
-
I saw that many duplicate id are referred to embedded image and lm files (that have also the same name). I dont think it is a problem, isnt it? I think that is not a problem if a node call the same embedded image or lm.
-
I have a problem
First I have changed all the duplicates Id and I replaced, in sub nodes and in the other files, the parental ID with the new ones.
I made an other try: I changed only the principal ID (command_room, galley etc), changing also the respective parental ID, but the result is the same.
Where is the mistake?
(https://s22.postimg.cc/kthfbno01/SH3_Img_11_3_2017_11_6_16_743.jpg) (https://postimg.cc/image/cb7z7bhh9/) (https://postimage.org/index.php?lang=italian)
-
I saw that many duplicate id are referred to embedded image and lm files (that have also the same name). I dont think it is a problem, isnt it? I think that is not a problem if a node call the same embedded image or lm.
That is exactly what I remember we found out when we coded the |WAC|-Dup-Node-Killer
There is no way to address bad data - such as this embedding is since stock.
PT and HS answered they would address this rest in manual work.
Maybe that manual polishing work still needs to be done. HS can tell.
-
I manually replaced the duplicate id but i have the problem in previous post.
I also tried to change only the main node id (and the parental id in the sub nodes) but the problem is still there.
-
Then state what is needed first, please
It is needed to remap all dulicated IDs... BUT... all related files (.sim,.cam,.dsd, etc) have to be reassigned manually.
All with S3D.. would take some dozens of hours!
Please give an explanation what exatly you have to do after our DUP-Node-Killer-tool has been run.
I would not do it with DUP node Killer... because manually youhave more control here with S3D.
If you let Dup Node Killer do all the work you dont know anymore where the parent ID from a related file (.sim file for example) belongs to.
An example here... Dup Node Killer changes the ID of the ID 1915 the therelated .sim file has got a wrong parent ID and several functions do not work anymore.
So.. every (.sim, .cam,.dsd) file has to be reworked with all the ne IDs given.
(https://abload.de/img/pick8ju0.jpg) (http://abload.de/image.php?img=pick8ju0.jpg)
-Short explanat