9/5/2018
Added filtering system to master spreadsheet
Great way to use the correct numbering system in Google sheets for sets/custom sets: https://productforums.google.com/forum/#!topic/docs/kl0Ueam1WfI (use a custom number format “000” so you can see the first game piece of a set as 001 instead of 1)
Will take a WHILE to get all the sets uploaded – no fast and easy way to get all the data into the spreadsheet the way it needs to be, especially considering the multitude of sources. (also having to copy and paste each ship’s cannons individually since they’re not in that format in the Jolly Roger or official WK spreadsheets) Will be difficult to do more than one set per day.
-Resaving a copy of the spreadsheet as an .xlsx document at the end of each day so I don’t lose everything if google implodes or loses the spreadsheet somehow XD.
9/3/2018 + 9/4/2018
-Added a plugin so that forum users at Pirates with Ben can now preview their own posts (you can also edit your own posts which I introduced earlier)
-Another MR up on Pirates with Ben (one per day)
-Updated all past MR’s with the actual stats and abilities of the miniatures in question (oops)
-Received some of the official set data/spreadsheets from Woelf. I will begin compiling, organizing, and formatting the start of a “master spreadsheet”!
(September 4th 2018)
I have begun making the master spreadsheet!! Today I finished Spanish Main Limited/First Edition. I’m using a combination of Wizkids official spreadsheets that Woelf is sharing with me, the Jolly Roger by Holofernes, the MT database, and various pieces of information from the Wikipedia page and MT forums for clarifications. SM Limited is actually one of the most confusing sets, so hopefully I can increase the pace as I go along. My goal is at least one set per day, but hopefully more. I am trying to be as comprehensive and detailed as possible, and once the spreadsheet is finally done I plan to make it public of course (though I may try to get a database up on my fansite before then, since I’d prefer to make necessary edits before a public release).
-These two final posts of this thread came in quite handy: http://m.miniaturetrading.com/im/forum/viewtopic/p=20282///#20282
9/1/2018 – A BIG breakthrough!
-Ran httrack on the MT database page for Spanish Main first edition to only take picture files (jpg, etc). “Get separated files” didn’t work, basically returned nothing downloaded. Then ran “download web site” with the same criteria (supposed to only download files with picture extensions like .jpg, .png, etc). This worked!! I now have all the picture files from Miniature Trading for Pirates CSG. The download returned more pictures than I bargained for, since I was after the max size pictures (a previous httrack run gave me tiny versions of them). It gave me three different sizes, but I seem to have the largest version of each picture in the Pirates database, which is great!
-Another MR up on Pirates with Ben (two actually)
SAVED:
All fleets from VASSAL Tournament #2 (T2) (so all of these fleets are saved)
8/30/2018
Did a search for “jpg” files in the File Explorer for the main httrack download that was completed some days ago. There are a TON of files and pictures (28,533 files for this one search!); I think I’ll make a zip folder with all the info publicly available in case other people want to browse it in the future for stuff from other games.
Another MR up on Pirates with Ben
SAVED:
Game Ideas and most MT threads related to it (not all custom set threads because some of the pictures are too small to see in a screenshot of the pages) https://imgur.com/a/yF3hU5L
I do find the timing of all this rather ironic and strange. Wizkids was shut down on November 10th, 2008. Miniature Trading may shut down in late November 2018, almost exactly 10 years to the day. O_O What should we prepare for in 2028? XD
8/29/2018
Edited forum at PwB so users can edit posts. (for the next 175 million years anyway lol, initial number was set too high I guess XD)
Got another Miniature Review in text form up on Pirates with Ben
-Tried two Chrome extensions that supposedly download all pictures from a web page – neither worked. One seemed to sign me out of my google account so I’ll probably stick with httrack for now.
-Used a search for “html” in File Explorer in an attempt to find html files of miniature pages (for Pirates of course) or forum topics. Haven’t found much of either, but there’s still a lot to go through. Truly a maze of stuff… some html files link to the same webpage, making it even more confusing. Plus I’ve got a ton of stuff from other (non-Pirates CSG) games.
-Generally struck out with trying new stuff. I still am on the lookout for programs or maybe extensions that give me a way to download entire forum topics at once somehow (such as a 50 page thread that would be very tedious to download pictures for). Feel free to comment if you’ve used something like this (repagination, downthemall, etc. are some I’ve seen)
MY PLAN
1. Get a searchable database at Pirates with Ben
-I already have picture files for most or all of the miniatures via a somewhat successful run of httrack. Most are not the full size so I hope to get the bigger versions of the pictures with more attempts.
-Woelf plans to share set spreadsheets with me, which should give me the correct versions of all miniature information. If there are any hitches with that, using the right scan settings/filters on httrack should give me html files for all miniature pages (haven’t found them yet in existing httrack downloads) which should be convertible to a spreadsheet from there.
–With those two items I should be able to create a database eventually, the how part is still to be determined.
2. Working forum at Pirates with Ben + Save info from MT forum
-The former is in progress but not nearly as high a priority as the database. Forum is up and running, posts show up, someone successfully joined and posted, threads and posts show up, etc – those are the main basics that are essential of course. Eventually I’ll mess around with settings to hopefully improve it a lot, better layout/look, hopefully change where the Login/Register buttons are, etc.
-Full page screen capture works great for saving threads from MT, but the picture files aren’t searchable like the text of course. I hope to find some kind of automated solution that gives me either full text from entire threads (Custom Ships, Rules thread, etc…) or pictures of the same, without me having to copy/save everything one by one. Not sure that httrack will help with this, but plan to look into other programs.
3. Trading system?
-No idea if this will even be looked at before October or beyond, but ideally between the database and the existing user profile format at PwB, I might even be able to implement some kind of trading solution as well. No promises because I’m basically still in the first stage of step 1! XD
8/28/2018
Fooled and played around with httrack for a while this morning – stuff I tried didn’t seem to work (trying to extract Pirate images from MT without downloading html), but I do seem to have most of the images on my computer now…
Put another of my Miniature Reviews on PwB as a text post
-Once again I looked at what httrack has downloaded for me from MT, and it seems I have html files for user profiles instead of the same for Pirates game pieces. Oh well, need to refine things and keep learning I guess.
[size=9]~~~~~
[i]Check out my fansite at [b][url=https://pirateswithben.com]Pirates with Ben[/url][/b]![/i][/size]
Daily update of the Miniature Trading crisis: http://m.miniaturetrading.com/im/forum/viewtopic/t=9350///
8/27/2018
Stopped an httrack mirror/download of the Spanish Main unlimited set url after 45 hours of it continuously running. (no idea why it took longer than the url for MT’s homepage, and it was definitely grabbing all sorts of stuff I don’t need)
Published another Miniature Review on Pirates with Ben (so I have the text and format in another place besides just a full page screenshot of the review at MT)
8/26/2018
Saved OTC3 thread https://imgur.com/a/HofdSiP
Made thread about unlocking stuff to edit posts to save raw text
Started posting updates at Facebook group as well
Saved the rest of my fleets – 116 in total! Took about 1.5 hours. https://imgur.com/a/jpcGciI
Uploaded pretty much all picture files taken so far to a combination of google drive/google photos (most were already on a combination of those two and imgur/postimages as well). In total some of the images I’m saving are going to have copies in 4 or 5+ places between my computer and the web.
Took pictures of threads from the ToT, album of forum images here: https://imgur.com/a/yF3hU5L
-Now all of the threads under “Very Useful Threads” have been saved.
I’ve always been interested in making things in Pirates more realistic to resemble real life because I’m so interested in Age of Sail naval warfare. If you’re like me, there are a lot of different things you can do.
(I put this together off the top of my head as a “rough guide” for people to look back at. Any ideas/house rules/threads/documents that you contribute would be much appreciated!)
– 90 degree move rule
No ship can turn more than 90 degrees when moving. In reality a ship wouldn’t simply turn around almost 180 degrees just because the sides of her hull didn’t overlap. (This is an old house rule that I use in my games, which can be broken when ships are otherwise surrounded and cannot move without breaking the rule.)
Since when did every cannon shot eliminate a mast? This is my personal favourite, although I’ve used it just once, for the Century of the Empires game.
– Instead of having derelicts sink after they’ve been hit once, they need to be hit twice as many times as they originally had masts. In this way, it takes 2 hits to sink a derelict one master, 4 for a derelict two master, and so on.
This really makes things more realistic because ships rarely sunk in combat (more likely burnt, blown up or scuttled).
There have been numerous ideas presented for this idea, many of which can be found in the linked thread (also this).
– Wind
As with return fire there are many different variations, of which I’ve found a lot on BGG (ctrl+F “wind” to find some of them. A bunch of them are very old and therefore have no thumbs ups. I was going to test at least three variations in January but I ran out of time).
– Treasure
It would be great if there was a way for players not to know how many coins were on opposing ships, since even this knowledge (without knowing the values) can be quite valuable. Other than an honor system I haven’t thought of a good way to make the gold game more realistic. As woelf plays, it’s good to use non-standard (30) totals of gold and keep gold face down on HI’s in all games. A friend of mine also suggested a box that would house all of the treasure, which is mixed up into different boxes and placed on islands so that nobody knows how many coins are on the different wild islands.
– Play with larger build totals
Normally I wouldn’t call a group of 1-4 ships a “fleet”, but rather a squadron (or duo/trio/etc. with less than 4). Most naval battles were not 1-on-1 confrontations but rather larger fleet actions between opposing navies. It’s difficult to make things actually realistic in Pirates, but if you want to try you can really jack up the build total. I’ve played three 5-player 500 point games (one of which was a deathmatch scenario), as well as numerous cumulative campaign games and even more games with a point limit of 100 or more. The biggest issue with this is the obvious time and space constraints, which have increased for me lately and sadly will only continue to increase in the foreseeable future.
This is a nice big list of different scenarios, some of which are based in historical contexts.
– Combine the naval aspect with a land/trade aspect.
This is something that comes up in campaign games, where the build total increases exponentially as gold is spent to launch more ships and hire more crew.
– Use proxies for historical ships from custom sets
You can create your own stats for your favourite historical ships.
Got email from user from reddit who recommended httrack – will look into details of stuff tomorrow
Ran httrack “update” or “fix” of some kind on the original mirroring process from MT’s homepage – seems to have updated the data on my computer, as the directory seems larger (way too much to look at/decipher)
Got this amusing message from the log file errors once it finished:
“23:00:30 Panic: Too many URLs, giving up..(>100000)”
-This second process took about 16 hours, very similar to the first. Might need to investigate running httrack on more limited urls (rather than trying to capture ALL of Miniature Trading lol)
Saved my oldest page of fleets, hope to save rest tomorrow https://imgur.com/a/jpcGciI
-I’ve been uploading pictures (of fleets/MR/forum pages) to a combination of google drive, imgur, and postimages for extra backup and security
Overwhelmed, but making progress! Hoping to make progress on various stuff every day.
