Replishot: Use Optishot with TGC 2019

[Zaren]

I have recently been working on custom software, Replishot, to capture data from the Optishot, process it and use it in TGC 2019. I've recently published my first pass of my program on github here. If you click on releases on the right there is an .exe you can download (Windows only). The page also has a readme that gives basic information on how to use the program.

Essentially the program captures Optishot data, processes it, and then turns it into mouse movements which TGC 2019 will recognize as a valid swing. The swing will change based on how hard you hit the ball as well as club angle, etc. The program works with the "video game" version of TGC 2019, not the "simulator" version.

It is very much a work in progress. I would appreciate feedback on improvements people would like to see. I am still working on getting more accurate ball flight based on what the Optishot sensors report, so if you're an avid golfer and the shots don't seem quite right... well, I wouldn't be terrible surprised at this point .

[Zaren]

BIG UPDATE! We're rolling to V1.4! (Note, latest release can always be found at the top of https://github.com/zaren171/RepliShot/releases).

Config File

The big changes include the addition of a config file! You can save your current config (using ball, always on top, etc.) and it will auto load the next time you run Replishot. The file saves with the name config.ini if you want to manually edit it. If you want to change the set of clubs in the dropdown menu, you'll have to edit it by hand. You'll just change True or False at the end of the line for a given club. Here's a couple example lines:

Driver=True
2Wood=False
3Wood=True
4Wood=False

If it says True then the club will show up in the list, if it says False it wont. Also, if there is no config.ini file, Replishot will use a "default setup" with a club set that matches the default club setup in TGC2019.

Front Sensor

The front sensor is now split into 3 regions. The top and bottom will change the club, and the middle will change the shot shape (pressing c key).

Lock Step Mode

This is probably the biggest change. You can now put Replishot in lock step mode. There are two main changes in lockstep mode. First, when you run your club over the front sensor to change clubs the club will be changed in Replishot AND in TGC2019. In addition, Replishot now performs OCR using Tesseract. What that means is that Replishot will now attempt to read the club selected in TGC2019 off the screen! This does use a bit more computing power, so if things are running slow you'll probably want to turn Lock Step mode off. Anyway, Replishot captures the top right portion of the screen and passes the screenshot through Tesseract which attempts to pull out all text there and pass it back. If Replishot finds a club name in the returned data then it will snap the club selection to that club! This means that when TGC2019 auto-selects a club for a shot Replishot should automatically update to the same club. Not the club you want? Swiping a club over the front sensor will change the club in Replishot AND TGC2019 when in lockstep mode. I'll note that the OCR function has a 1 second sleep between calls, so it can take a few seconds for results to show up. If Replishot is not automatically updated to match the TGC club I find that change the club (swiping over the front sensor) to different clubs can sometimes help. Once OCR catches a club and Replishot and TGC2019 match again then front sensor can be used to get back to the club you want for your next shot. Play around with it a bit, it's great!

Conclusion

The latest update of Replishot brings some big changes. Things are getting close to the point where you ideally won't have to interact with the Replishot program, and can even do most of what you need from the Optishot without needing to touch your computer! I'm still working on a good method for aiming left to right.

The readme contains (or at least is supposed to) additional information to what I've shared here, so feel free to give it a read and hopefully it'll answer any questions. If you do have a question, please ask!

[hen7]

I'm like a kid in a candy store! Stoked to try these new updates!

Thanks again for all your hard work - you got some ninja moves in your bag!

[hen7]

I think you may need to have an option to remove the front sensor club selection.

7 out of 10 times I swing, the club changes in TGC right after I swing, followed by a swing with the wrong club in TGC. It's not usable at this time.

Thanks for trying to figure this one out. You can try swiping your hand across the optishot for testing, it gives similar results.

[hpdrifter]

That does sound like it would cause a lot of club changing. I dilly around too much around the sensors.
Maybe if you could turn it on by swiping club long wise down the sensors or something.

[Zaren]

The club changing is likely not actually caused by the front sensor. There's a bunch of stuff hidden under the hood. Part of taking the shot is to automatically apply a shot shape, so the program should automatically add in a hook/slice depending on how your club came through. However, if you haven't changed clubs since the previous shot, the previous shot shape is still there, and the next shot would add to it. Replishot is actually supposed to press z and then x before each shot to cycle down a club and then back up one. Changing clubs removes the shot shape and returns to a default straight shot. For the life of me I can't figure out why these key presses get missed sometimes. When I first played around with it it seemed the first key press was the one getting missed on occasion, so I added a "W" press at the start as w doesn't do anything in TGC2019. Things have been working well on my computer, but apparently there's still an underlying issue here. When playing a round I think the shot shape will reset between each shot, so maybe I need to add a driving range option, and only at the driving range it does the club swap.

I can also add a toggle for the front sensor, but I don't think that's the problem in this case. I'll explain my reasoning briefly as I don't want to dive too far into the Optishot data format. The Optishot returns 3 types of data. Back sensor only, front sensor only and both. The back sensor only is used in the optishot software for calculating the swing tempo. It also can't really be used for anything special because a normal swing will trigger the back sensor only every time you pull back, so if a club change was triggered by the back sensor you'd change clubs before every swing. When you get data from both the back and front that's a swing. In order for the optishot pad to return data from both sensors in one data packet the club has to pass over both of them close enough together for the optishot pad to group the data as a "swing." A swing is never 2 packets, one from each sensor, but is always a single packet with data from both. Finally, you can also get front sensor packets, which I process for extra features.

Thanks for the valuable feedback from your testing. I'll see if I can at least get the "driving range mode" put in today and pushed out.

Out of curiosity, does Replishot apply a shot shape in TGC (a red line appears when the hook/slice is applied on the driving range, it's pretty hidden on an actual round, but you might notice the shot shape box appear)?

[Zaren]

Replishot v1.4.1 is now released. Thanks for the feedback! Under Options I've added Driving Range Mode as it seems that only in the driving range do you need the club swap to clear a previous shot shape. I did some testing in game and when the game suggests the same club on back to back shots the shot shape still reset to a straight shot. I've also made the front sensor features optional (and off by default). Hope these fix some of the issues seen with the v1.4 release.

Thanks for the feedback!

[Zaren]

Nah, it's fair to make the front sensor stuff optional. It was a good point. In my personal use I don't think I've ever accidentally triggered just the front sensor, but that doesn't mean everyone will want those controls always on.

[hpdrifter]

I got another question for "eventuality"(when I acquire TGC): Will a remote box for Optishot control/selections, as discussed in another thread here, be operable with the use of Replishot. Just curious in case I get the urge to try and build a box.

[Zaren]

Replishot itself will not forward the keyboard inputs. If you build the box yourself then at some level you'll need to be assigning which keyboard key goes with each button. If you're doing that assignment you could instead match the keyboard keys to those used in TGC. You'll have to keep TGC as your selected window (meaning it's the last thing you clicked on). The hope with Replishot is that in the end you don't actually have to click on and interact with the Replishot program, keeping TGC in focus and all inputs going there. I think it's nice to keep it on top to see swing parameters, but ideally it'll just be there to show show information, capturing swing input data and turning it into mouse/keyboard commands for TGC, and any interactions done by the player would be done in TGC (such as aiming, club selection, etc.).

The effort I've been putting into the front sensor extra features is to try to allow the Optishot to be used in place of a remote box. A remote box (for Optishot at least) typically has buttons to change clubs, aiming left/right, and changing the loft of the shot. Currently the front sensor can be used to change the club and shot "type" in TGC. The only missing feature currently would be aiming, and then the front sensor features would hopefully be sufficient enough to replace a remote box for most instances.

[hen7]

Hey Zaren,

Thanks for making such a quick adjustment! It's great to have the option to turn off the club selection as it seemed to be interfering somehow(I'm not sure - I'm not a programmer at all!).

I do see a red line now, and I don't think I saw it in 1.4.

I haven't had much luck with the lock step, but will experiment more over the weekend. Happy Easter!

[hpdrifter]

Zaren, seems like you've seen the 'inner workings" of Optishot.
I'd like to know how Optishot decides "open or closed" on the club face.
I thought it might be an average calculated "position" based on entrance angle and exit angle.
I've calibrated, recalibrated, and have adjusted the tape on my driver and 3 wood. It persistently say it's open and by large degrees.
If it were open, the ball would take off toward the open clubface in real life into the net with an onplane swing, so that indicates it is NOT open.
Plus, I can see that it's not open.
I have a dark space with the only light over the sensors being led.
I know it's open coming into the sensors on the back side of the ball, but it is closed coming out of the sensors.

[Zaren]

I cant comment on how the Optishot software itself does the angle calculations. I know how I did it. I don't want to throw out the exact data format the Optishot coughs out (in case there's a reason I shouldn't), but from the data I am able to "calculate" a club face angle. That assumes that every sensor fires correctly for the club head, which it does not always do. I was using the Optishot software a week or so ago and I calibrated my club and about halfway through a round I started getting crazy shots and mulligan after mulligan could not get a normal looking shot out. I deleted my club calibration and suddenly my shots started looking normal again. I don't know what's going on, but I do know that from the data I receive it's possible to calculate a club face, but you have to trust that the sensors are capturing the correct data, and one sensor missing the club face can radically change the angle.

[hpdrifter]

Fair enough. I didn't really expect a down and dirty explicit reply(for legal reasons) Just mainly if it used front sensors only, rear sensors only, or a combination of both.
Thanks for the reply.

I put some of that "reflective mailbox tape" on my driver and that stuff really reflects. I can swing the driver 4 or 5" above the sensors and it'll pick it up.

BTW: how do you delete a club calibration? Just re calibrate?
Edit: finally opened my eyes and saw "Delete" in calibration window. duh!

[breezyb]

Hopefully not too late to this convo. Looking into TGC and found it on steam but is that just the video game or the program we are talking for simulation here? Is there any difference. I did some googling I promise but the answer is surprisingly hard to find.

[Zaren]

Replishot specifically is designed to run with the TGC 2019 video game available on steam. It uses the data collected from the Optishot and does it's best to control the mouse/keyboard to approximate the shot recorded by the Optishot. I'm not surprised Googling didn't provide a clear answer on that as Replishot is pretty small and recently released. If you go to the github page (https://github.com/zaren171/RepliShot) you'll see that in the readme there is a link to TGC2019 and it points to the "video game" version on Steam's webpage. You don't have to get it from steam per say, but you do NOT want to buy the ~$1000 simulator version.

[breezyb]

Awesome! Thanks for the response Zaren!

[hen7]

Hi Zaren,

Any tricks to making the lock step work? I haven't had any luck (no change in replishot club). Do I need Tesseract software?

Thanks!
Dave

[Zaren]

Nah, Tesseract is all baked into the Optishot executable (that's what increased the .exe file from ~2MB to ~25MB). Tesseract is only checking the top right portion of the screen. If you're running multiple monitors it might be grabbing the wrong screen. I don't have multiple monitors, so I haven't tested that. I don't grabbed the full screen to try to cut down on the processing time. Tesseract should be reading the info box as seen here:

https://github.com/zaren171/RepliShot/blob/master/TGC_Shot_Settings.PNG

so make sure Replishot isn't sitting on top of that. Also, some clubs don't seem to read as well, so I'll sometimes change the club (I use the front sensor features) and after I change the club a few times things usually lock together, at which point you can change back to the club you want to use.

Hopefully any of that is helpful. If you keep having issues let me know.

[hen7]

Hi Zaren,

Thanks for your reply, but I'm still having no luck. I did notice that I'm unable to keep the dropdown menu open for club selection (flips right back up), so I use my up and down keys once the box is highlighted. Maybe there is an issue there. I'm using win10.

I was running a mirrored screen, but I tried single screen with no luck.

Thanks for any other suggestions.

Cheers, Dave

[hen7]

Sorry, I didn't take your earlier suggestion of playing around with the optishot sensor selection. I got it working now I think. It's great!

[Brettster]

What exactly did you do that worked hen7? Was it the Tesseract reading the screen area and how did you finally get it to work? Would be good to have here, for others that might want to get it to work...

[Zaren]

To enable the Tesseract code you have to turn on the lock step function under the options menu. Make sure that "Lockstep Mode" has a check box next to it. You can click it to toggle it.

What I suspect the issue is, is that sometimes Tesseract fails to read the club off the screen. When it processes the text it will come up with some incorrect word other than "driver" or "iron" and so my software doesn't see a valid club name. Tesseract is really intermittent with this, so as an example sometimes it'll read 3 Wood just fine, and sometimes it doesn't. A work around it to change the club selection. I use the front sensor features so I don't have to go to my computer to cycle clubs, but if you change the club in TGC2019 Tesseract will pick up on a different club's name. Once Tesseract correctly reads a club name, Replishot will select the same club. Now that the clubs are in sync, changing the club with the front sensor will change the club in Replishot and TGC2019, keeping things in sync without Tesseract's intervention.

The goal of using Tesseract is that TGC2019 will automatically select the "best" club for you, and there's no good way for Replishot to predict what TGC2019 will pick, so Tesseract is being used to try to keep the clubs the same. It's pretty trivial in Replishot to change the club in both locations, to hard part is keeping them in sync when TGC changes to a random club.

Hopefully some of these insights are helpful to others who are trying to debug. If you don't to use the front sensor features you could also just change the club around in TGC using your keyboard until Replishot locks to it. And as a final note, tesseract runs once per second, so you might have to give it a few seconds to lock.

If you have any questions, please let me know.

[hen7]

Hey guys,

Sorry, I think I may have spoke to soon. I don't think the Tesseract code is working correctly, or at all. I have to first manually line up both the TGC club with the replishot club, only then I can use the front sensor to adjust the clubs simultaneously. I thought it was working at this moment, but as soon as I move to the next shot, TGC has auto picked a new club, and Tesseract hasn't been able to pick that new club up. From here I have to manually match the replishot club to the TGC club before continuing. So with this being the case, I have found it, well, easier to just play with the Optishot software for now. I feel like Replishot is close, but just needs a way to remove the user from having to go the the computer for every shot.

There is an option in TGC to not auto select a club, but it defaults to a driver, or a putter, which unfortunately doesn't help. If it could somehow select the last clubs used, the clubs could at least stay in sync.

Side note - The drop down menu for club selection in Replishot has always given me trouble. When I click anywhere on the drop down menu, it only shows the list of clubs for a split second, and then auto closes. From here I have to use my up/down keys to select my club.

Would there be anyway to pull code from TGC to know what club is being selected on screen? I'm sure you've already thought of this though.

Thanks for your time on this cool piece of software.
Dave

[Chuck]

I was wondering if anyone knows how to remove the avatar player when playing a round. Is there a variable or a setting I'm missing? Thanks.

[Zaren]

I poked around the settings and couldn't find anything either. I'd guess the issue is that with this being the video game version, you're meant to be controller a player, and not magically launching the ball. It's too bad, that would be pretty neat to hide the avatar.

[hen7]

Good news - I finally figured out my issue with the club selection. It was my multi-monitor setup. If I have just one monitor hooked up, it works great!

One issue I have now is that I can't seem to putt more than 10 feet, even if I swing my putter. Any ideas on how to fix that?

Thanks!

[Zaren]

Hum... well my first thought is to make sure Replishot has putter selected. The putter has to be held back MUCH longer than other clubs to get distance, so it has very different code when it's used. If Replishot is using a club other then putter then you can't put very far. This is because to get a good swing with other clubs you have to keep your backswing within a certain tempo, and can't go too slow. But for the putter, the longer you hold the mouse back the further the putter is drawn back in the game. I can take a look at it, but I don't think I've had any recent issues with putting that I can remember.

Glad you were able to figure out the club reading/tracking!

[Fore on the Floor]

Hello Zaren,
Do you think this could be modified to get the Opti shot to work with other software other then TGC? I haven't played with this yet (my opti shot arrives today) but I am looking forward to it. I am really trying to get Creative golf 3D to work with a usb launch monitor.

[Zaren]

I believe so, yes. In my poking around it looks like most of the launch monitors out there use JSON formatted data over a network connection. At it's heart, Replishot is essentially just a custom program to grab data from the Optishot sensor and process it into a usable format. When I say usable I mean turn raw data into things like club head speed, path, and face angle. Further processing and estimations could turn these into approximate carry distances, etc. which could then all be packed together into the needed JSON format and sent out over a network connection. Some of the launch monitors have easy to find documentation that explain what the JSON formatted data needs to look like, so assuming you can find an API for one of the launch monitors supported by Creative Golf 3D, there's no reason you couldn't translate the Optishot data into the data needed by Creative Golf 3D.