Introducing TGC Designer Tools and Real Course Recreation Lidar Import

[ChadGolf]

Hi everyone!

I've been hard at work for the last month developing tools that primarily will import Lidar data into TGC 2019 Courses.

There have been a lot of ups and downs, but I'm at the point where I'm ready to accept feedback from the community.

A completed course file is available here:
https://drive.google.com/file/d/1RYW...ew?usp=sharing

Please don't publish this course in game, it's not completed and I call 'dibs' on publishing this course after I finish decorating it.


I've completed the proof-of-concept phases and have created playable, realistic courses. I'm still working on the User Interface and instructions to make this the easiest that it can be, but I wanted to open the tools up for developers to contribute fixes.

The source code is available free on Github and releases will be available there under the "Releases" link. I'm targeting the initial release in a week or two as a Windows download:

https://github.com/chadrockey/TGC-Designer-Tools

https://i.imgur.com/4GnzENd.png

I also am supporting import from OpenStreetMap golf courses ( https://www.openstreetmap.org/#map=1...576/-115.28536 ) into TGC as splines if you don't want the lidar elevation or can't get lidar for your course.

https://i.imgur.com/Nza71FG.png

Quick tips if you want to get started on your OpenStreetMap course: use the highest quality sat image available, map greens along the inside of THE FRINGE, map bunkers on the outside lip, map water on the outside lip.

For more information on the process or if you want to support future features of this tool, please read here: https://www.patreon.com/chadgolf

I hope this brings enjoyment to everyone. We all have courses that are personally meaningful to us, and I hope this software inspires you to create something for the community.
- Chad

Special thanks to MTHunt, crazycanuck1985, friendBOMBER, and HTdumps for initial feedback and guidance.

Instructions and a tutorial linked from this page: https://chadrockey.github.io/TGC-Designer-Tools/

[BGCurtis2nd]

ChadGolf Seems if there are a lot of trees it thins them out but individual trees it seems to be right on, to me it is a lot easier to add trees in where you have a marked layout for a group than it is trying to find the right point for an individual tree. I love it, might have to do a bit of changing over on type and size but for what little I have played with it this morning it is a big time saver, trying to cut down my map a lot closer into the golf course had 6919 trees on my first run.

[TFLeonard]

I want to add my thanks to all the effort and work going into helping the RCR community. I'm trying to recreate some golf courses in Normandy, France (near the D-day beaches etc.) and its been pretty tough to find Lidar data in a format that can easily be used in TGC Golf Tools or anything else I have tried. I'll post a separate message regarding files that I've found, but I've been tinkering with creating a "fake" LAS file by converting STRM data to a LAS file (I'd upload a copy of my .LAS file, but this board won't allow non-image attachments). I've attached the results and as you can see, the terrain doesn't line up with the OpenStreetMap plot. As a guess it looks like maybe the terrain projection is off. Would I need to add a EPSG number? If so, what would that be?

This particular course is "Golf Cabourg-Le Home" and can be searched on OSM. I realize STRM data is not fine enough to really do the course justice, but having some sort of elevation map to plot a course is better than none. I used to manually raise all the features by referencing Google Earth, and then filling the gaps in course designer, but that's very labor intensive, so if we could automate that even with a fudged heightmap like I'm attempting it could help a lot.

On a side note, would it be possible to use a grayscale bitmap as a height map? That would make it easier for non-programmers to edit and produce the map.

[ChadGolf]

That angle difference is definitely a misprojection, try a few different EPSG until you find one that is consistent.

Its further down the list but STRM, DTM, and Geotiff are possible to add support. The main downside is that these are already made in a way that might remove bunker or green contours and the resolution is usually lower than the original lidar.

Having 1/9 or 1/3 arc second resolution is probably better than nothing at all.

[TFLeonard]

I've found a site which seems to have Lidar data for France: http://ids.equipex-geosud.fr/web/guest/france

I'm not sure how to make use of it since the data seems to be in csv text. Here's a sample:

G2	G2A	G2Std-dev	G3	G3A	G3Std-dev	G4	G4A	G4Std-dev	G5	G5A	G5Std-dev	G6	G6A	G6Std-dev	Signal start(3.5)	Signal end(3.5)	Signal start(4.5)	Signal end(4.5)
113	0.06	3	135	0.1	6	165	0.15	9	221	0.1	42	332	0.04	3	0.95	386.24	8.3	384.17
305	0.42	17	317	0.14	4	331	0.15	3	338	0.31	12	372	0.06	4	22.22	403.24	24.51	399.16
171	0.03	2	185	0.11	3	203	0.07	4	235	0.07	20				0	423.23	0	422.08
131	0.05	6	172	0.08	12	195	0.02	2	212	0.07	4	493	0.02	2	41.7	503.25	42.83	494.9
450	0.44	14	471	0.55	6										394.37	493.3	403.28	486.85
470	0.49	8													405.55	501.39	406.83	492.66
462	0.68	13	486	0.04	2										392.9	541.38	407.44	493.73
431	0.69	14	447	0.04	2	456	0.14	7	469	0.14	14				363.76	521.16	372.98	507.28
481	0.31	8													431.3	503.48	434.08	501.53
457	0.41	14													412.24	500.41	417.88	496.99
439	0.25	6	454	0.25	17										406.01	515.94	413.54	500.71
459	0.11	12													373.2	509.99	387.82	491.47
429	0.6	6	451	0.27	8	470	0.08	7							390.63	506.99	395.29	500.46

Can this be converted into something that TGC Golf Tools can use?

[TFLeonard]

P.S. there's also a whole series of smaller .csv's that have this format:

Raw waveform	Smoothed waveform	Sum of Gaussians
0.045021	0.03016972	0.018008586
0.051696	0.040035726	0.01885829
0.051696	0.046767244	0.019736342
0.045021	0.046192775	0.020642993
0.038346	0.044688186	0.021578455
0.045021	0.04360757	0.022542896
0.045021	0.043849225	0.023536437
0.045021	0.045021	0.024559153
0.045021	0.048536325	0.025611071
0.045021	0.056291942	0.026692163
0.065046	0.06738955	0.027802348
0.085071	0.078970469	0.028941487
0.098421	0.085048164	0.030109386

Are they of any use?

[ChadGolf]

Sorry, that's pretty raw data. If it's not in X, Y, Z or at the very least lat lon, it's going to be hard to use. We'd have to figure out the path of the airplane to use data in this form.

[BGCurtis2nd]

ChadGolf I have been playing with the new 2.0.0 and have to say I do love the LIDAR tree import, may not be perfect but sure does beat trying to set them in any other way I have tried. They seem to be close to the size needed as to where using openstreet gives you one size and from what I have been doing all to large. Just thought I would ask if there could be some way to do a mask on the first selection to cut down on the trees oust side the course, box selection seems to get a lot more than needed, could you set up an area select by chance? Believe me, I am not complaining just asking you have done a great job on this and without it I don't believe I would be trying to build courses.

[jrz]

What OSM tags are you looking for for the new features such as trees?

[mp1972]

According to an earlier post from Chad:

natural: tree (Point Type) -> rendered as the default tree/default size for your theme
natural: wood (Area Type, think forest) -> rendered as a gravel spline
ALL building: tags (including golf:clubhouse) -> rendered as a gravel spline
ALL waterway tags (as long as they don't extend outside the course boundary, maybe map them in shorter lines) -> rendered as mulch splines, but do not have to be areas

[jrz]

Thank you. This thread is getting huge.

[mpendle]

ChadGolf

Just in case this is missed as an earlier comment....

In 0.2.0, I'm getting a lot of these errors when loading UK Lidar to existing courses I'm working on.
Proj4 : pyproj.Proj('+units=m +init=epsg:27700 ', preserve_units=True)
Processing: TQ4260_P_9893_20150303_20150305.laz
Using PRJ file: LIDAR_point_cloud_tile_catalogue.prj
Could not load TQ4260_P_9893_20150303_20150305.laz Please report this issue.
Processing: TQ4262_P_9893_20150303_20150305.laz
Using PRJ file: LIDAR_point_cloud_tile_catalogue.prj
Could not load TQ4262_P_9893_20150303_20150305.laz Please report this issue.
Processing: TQ4264_P_9893_20150303_20150305.laz
Using PRJ file: LIDAR_point_cloud_tile_catalogue.prj
Could not load TQ4264_P_9893_20150303_20150305.laz Please report this issue.
Processing: TQ4266_P_9893_20150303_20150305.laz
Using PRJ file: LIDAR_point_cloud_tile_catalogue.prj
Could not load TQ4266_P_9893_20150303_20150305.laz Please report this issue.
Processing: TQ4268_P_9893_20150303_20150305.laz


Any idea what I should do?

[mpendle]

Entering the EPSG directly has worked - just strange that I didn’t need to do that before (on the same course).

[ChadGolf]

It's not unknown to break something. I'll look into this tonight. Can you access prj2epsg.org in a browser?

[mpendle]

Sorry for slow reply, yes I can access that webpage in a browser.

[vrpunk]

I’m on my iPad away from pc so can show you links.
this is the link to the metadata

https://ea.sharefile.com/share/view/sd05f5c0e9d54e1d8

which I got from here

https://ckan.publishing.service.gov.uk/dataset/lidar-point-cloud

[Laboroflove]

ChadGolf great tool! Just published my first beta course LOL Test 1, which is a RCR of George Wright Golf Course in Boston MA. Now I’m onto course #2. I’m going after courses that have high accuracy Lidar data. I’ve noticed a bunch of older data out there, but have seen in the Boston area that there is a lot of 2018 data that is about to be released. I’m gonna hold off on some of those courses until the newer data is available. Leads me to project #2:

Newport Country Club, host of the 2019 US Senior Open and one of the oldest courses in the country. I’m also trying out your tree tool this time, and since there are not many trees on the course(it’s next to the water) I’m thinking this one is going to go very quickly!

one thing to share, my first file was getting a network error in TGC when I tried to edit the course. I pinpointed it to a tree that I tagged in OSM with natural:tree. There was also a Lidar tree right on that same spot. As soon as I deleted the tree tag from OSM and reran your tool the network error went away and everything else worked. Auto trees and tagging the woods is excellent!

[ChadGolf]

Network errors are when the course crashes the game! XD

Yeah, you're not supposed to use both OSM trees and lidar trees. Let me know if ONLY OSM trees or ONLY lidar trees work. I'll see if I need to strictly enforce this somewhere.

It would be funny if things were so accurate that the OSM tree and the lidar tree were on the same spot and made the game unhappy because of the accuracy!

[mp1972]

@ChadGolf

Not sure if this is exactly the same thing, but I'm pretty sure I was able to load up lidar trees and woods from OSM. I didn't have any trees marked, but did have at least one wooded area marked and it worked without error.

lidar trees is unbelievably cool btw...

edit - oh, I see this is different. The woods area just put a gravel area there while the trees I am seeing are from lidar. Was think marking an area "woods" would fill the areas with trees, but not the case. still cool to see the 2 things overlap correctly.

[ChadGolf]

Yeah filling splines with brushes isn’t easy at all. I had to make an algorithm to brush in all the terrain and I don’t think I’ll do that for aesthetic features.

Just making progress bit by bit.

[theclv24]

Hey ChadGolf, I imported OSM data into a flat plot this morning and it worked great, but then I saw a new release for the tool so I decided to try it again and see if anything changed. Now when I start with the same empty course file and the same OSM export file, I am getting this error message:


I've tried turning off various Import elements and seeing if that changes anything, but I get this error message every time. Any thoughts?

[theclv24]

Lahinch, Ireland. The main target is the Old course on the West side, but I also laid down the Castle course (East side) with no holes marked, as it will be visible.

[ChadGolf]

Thanks! I found the issue. I needed more information to do the OSM tree import, but I was still only sending a limited amount for the flat course import.

I hope you had an older version around and weren't held up. I'm committed to leaving the older downloads up and clearly labeling each one so if something breaks or someone likes the way it was done before, they have that option.

This will be fixed soon, I'll probably make a release just to fix this since it's a critical function. I can't promise to never break things, but I'll do my best to fix them or allow members of the community to fix issues.

[theclv24]

No worries, this project is a little down the line anyways. Thanks for the update!

The other issue that I will want to hit you with at some point is that I had previously been working on this course before the tool was available, and put a lot of time into creating the coastline and town. I would love to slide the OSM import file directly into that plot and try to line things up. In my first attempts to combines directories, however, it was having an issue importing my previous course. I will grab a screenshot the next time I give it a try.

[vrpunk]

Greens Putting Book.
My goal is to produce a Greens Putting book from google earth + breakmaster (digital green reader) for our local course at the new 2019 rules scale 1:480.
Then the LIDAR thread came live....and it got me thinking

ChadGolf as a bi-product of your tool, using the OSM+Lidar can we isolate the putting areas, then use the Z co-ordinate to produce a numerical bump map on the greens either representing changes in height or another unit of measurement (Degrees or %).

How cool would it be for an end-user to produce a 18 page Greens Book PDF from the OSM+Lidar with a simple checkbox option.(No TGC required)

Maybe its not possible or is a lot of work but all I would be doing manually is similar to OSM (drawing the green shape), LIDAR (using breakmaster at various points to find my elevation)

Let me know your thoughts.

[ChadGolf]

It could work, I'm not sure if it's entirely accurate enough to make true green books. It's usually decent enough to get the direction right in lidar, but I'm not sure every undulation would be available. Depends on the quality of the lidar and the density. I know there's at least a few businesses that go out to golf courses, take up 2 tee times, and instead of playing the holes they move lidar equipment over the greens. This gets you 1cm accuracy with essentially infinite density. Even the best lidar I've found is 0.7m between each lidar point.

Feel free to fork the source code and see if you can make this work, the basics are all there: heightmaps at various resolutions from lidar, annotated with open street map.

[RaceTo100]

ChadGolf. I have a thought for trees that might be kind of a cheat and produce the best recreations. It will still require digitizing.

If there was a way to import a csv file with x,y,z, tree_type into the course designer instead of LiDAR and OSM. Many of the trees are huge areas where there are tons of trees where you could use the multi add, but say a couple hundred are right along the edges of the course. Maybe right in sight lines. Getting those exact would be amazing. Then it’s up the the individual to digitize them and they could import. Just a thought.

Thanks again for everything. I hope this post is in the right spot. I’m using my phone and don’t normally use my phone.

[ChadGolf]

I'm all for any idea that saves time. Do you happen to know of a service that already provides tree locations?

The two implementations I've done now rely on:
You already have ground classified lidar, so let's add trees from lidar peaks
You went through a lot of work to georeference your lidar from sat images on OpenStreetMap, you could click and manually add trees that way.

In the game we only need the 2d coordinate position of an object, we can set it to "ground height". So if there's a way to get lat/lon tree positions, I'm all for it. I think lat/lon would be the only coordinate system that would be portable enough to use in all situations.

[frodaddy00]

Plotting your course in OSM - 2 hours
Finding lidar data - 1 hour
Using Chad's awesome tool to create course - 10 minutes
Adding 10 billion trees by hand - 50 hrs and counting..

Its too late for me to use the cool new tree feature since I'm already heavily invested.. trees trees everywhere.


Here is a flyover I did of the front 9 that I'm almost done with (except some missing bridges).. now to finish up the back 9. I wish there was a way to make my woods thicker as far as ground cover goes..all I could think to do was add bushes...trying to recreate how penal the course is on wayward shots instead of just easy punchouts back into fairway.

https://youtu.be/Dyd31SUWVJ4

[Fhacker]

ChadGolf. Yes agree it's time consuming but was just offering a suggestion to frodaddy about a possible solution. And if u succeed in more auto import of trees, other features, etc., we may not need designers anymore. They won't have anything to do but layout tees/pins etc. Ha Ha. My compliments to u again for this amazing accomplishment

[ChadGolf]

I'll never do bridges or animals.

[Fhacker]

ChadGolf. LOL

[RaceTo100]

Plotting your course in OSM - 2 hours
Finding lidar data - 1 hour
Using Chad's awesome tool to create course - 10 minutes
Adding 10 billion trees by hand - 50 hrs and counting..

Its too late for me to use the cool new tree feature since I'm already heavily invested.. trees trees everywhere.


Here is a flyover I did of the front 9 that I'm almost done with (except some missing bridges).. now to finish up the back 9. I wish there was a way to make my woods thicker as far as ground cover goes..all I could think to do was add bushes...trying to recreate how penal the course is on wayward shots instead of just easy punchouts back into fairway.

https://youtu.be/Dyd31SUWVJ4

Yeah I had some issues with my LiDAR classifications. I used another program to night to really clean up the high vegetation and wow. Simply amazing. If the trees are coming in strangely, it’s all in the LiDAR classification. Just wow. So amazing.

[ChadGolf]

Tell us more about this other program, friend.

[Genghis]

Yes, please. Will it run a classification routine on a LAS file where none of the points were never classified?

[ChadGolf]

A small new release tonight, fixes a few issues that everyone's reported - thank you for pointing those out. The lidar trees will also respect the mask now.

Full changelog and download available on the release page that's listed here on the project's homepage: https://chadrockey.github.io/TGC-Designer-Tools/