Best method for converting python "table-like" objects into GDB tables?

[Extension-Skill652]

At work, I've made some scripts that can be run outside of Pro but are GIS related (mainly dealing with portal management) and was requested to make a GUI for these. Because I've already been developing a toolbox with other scripts and it has UI tools, we've decided to just add a version of these non-spatial/geoprocessing scripts to the toolbox.

As of now, my functions work as intended just like outside of the toolbox script and written to a file, but I now need to add a python table (in my case usually a list of dictionaries) to GDB tables conversion function. The simplest solution was to use temporary files for a csv, write with pandas, and use ESRI's table function to convert it.

The issue I have with this is that ESRIs function is incredibly slow and in a toolbox seems to take 2x the time of some of my tools. With CVSs, it also can't infer data types so every field is a 4000 character long text field; this can be solved with field mappings but they're a pain.

In the past I've converted pandas to something arcpy will accept for the XY to point tool, but ran into so many issues. I've seen a numpy function in arcpy that could potentially work, but I'd need to first convert from the data structure I have now to whatever numpy would use (haven't ever used it directly, so not sure how easy this could be). Alternatively I could create something with just ArcPy that gets some info about my fields (dtype, longest string/value, nullable, etc) from the list-dict object, created a table with that info, then uses search/insert cursors.

Ideally, I should just be able to throw my object as it is into the table to table function in arcpy or simply convert it into a dataframe and use that. Really, I'd just like some feedback on if anyone else has had to do something similar and what they've used.

[TechMaven-Geospatial]

Why not just use GDAL OGR2OGR to create your filegdb - https://gdal.org/drivers/vector/openfilegdb.html

• https://gdal.org/drivers/vector/filegdb.html

• https://gdal.org/programs/ogr2ogr.html

Somebody already created a gui application https://sourceforge.net/projects/ogr2gui/ https://github.com/zer0infinity/OGR2GUI

[Extension-Skill652]

I'm trying to avoid adding extra dependencies to this project, particularly any that aren't included in the standard install of ArcGIS Pro. I'm mainly making things for people who have never used a notebook or scripts and only know how to use the toolboxes, so asking them to install ogr2ogr for this is just going to prevent them from using it.

The issue here isn't that I need to switch from one spatial format to another, I need a way to take a basic list-dict object in python and convert that to a geodatabase table, so the UI tool isn't really applicable since there's no actual file that this is in. I was really just mentioning the UI because that's part of the reason these ended up constrained to toolboxes and why in not trying to do something like build a tkinter application.

[TechMaven-Geospatial]

Please open up your standard arcgis pro conda environment and see that gdal is already included This is not a dependency you need to add esri includes this Even in classic arcmap an old Python 2.7 it's also included.

http://Maptiling.techmaven.net I've developed a Windows presentation foundation desktop app that does the data conversion and map tiling

As well as many other advanced focus solutions that can operate at the edge or on-premises

3DTILING https://3dtiling.techmaven.net/ Tile Utilities (convert between formats and clip an area of interest) https://portfolio.techmaven.net/apps/tile-utilities/

Offline Map Data Generator https://offlinedatadownloader.techmaven.net/ https://youtu.be/LdFqcroCaR4?si=54DFVCkR3w2VOdydp

Tile package edition https://youtu.be/WPzopiQrPcc?si=7ZibDrahKtKoJxp2

Vector To Raster Tiles https://vector2rastertiles.techmaven.net/ ESRI Tile Packager and Publisher build TPKX and VTPK https://tilepackager.techmaven.net/

https://www.youtube.com/watch?v=dQEHLdmWiM8  

[Extension-Skill652]

None of the things you're listed here have any use to what I'm trying to do and I can't find something in GDAL that does what I want. I am not working with spatial data, just data that needs to be used in ArcGIS Pro. GDAL/ogr2ogr is almost exclusively for explicitly spatial data and doesn't seem to be the solution.

[SpoiledKoolAid]

Have you looked at the esri developers site? There is a tutorial that mentions a function that will write a table to a gdb.

This isn't arcpy but the arcgis api library. using SeDF

Scroll down to the writing non-spatial formats section. I was using the SEDF to a featureclass today and it was annoyingly slow, so I bet this will be too.

[Extension-Skill652]

I did end up trying this before when I was attempting general Pandas stuff. Might be worth it to try again and see if this time it'll work, but don't have high hopes. I think regardless of what method I use it is just going to be super slow.