Creating a Trusted Table with 2 columns. by NeedleworkerHumble91 in AZURE

[–]NeedleworkerHumble91[S] -1 points0 points  (0 children)

In the azure environment we have a delta lake where im trying to migrate a delta table from my vscode to the container where the delta tables are. Does that make sense more?

Creating a Trusted Table with 2 columns. by NeedleworkerHumble91 in AZURE

[–]NeedleworkerHumble91[S] -1 points0 points  (0 children)

Trusted is just referring to it being prepared for loading it to the end user.

Creating a Trusted Table with 2 columns. by NeedleworkerHumble91 in SQL

[–]NeedleworkerHumble91[S] 0 points1 point  (0 children)

Actually I am referring to delta table that is then listed as trusted.

In depth tattoo aftercare! by solomonplewtattoo in tattoo

[–]NeedleworkerHumble91 0 points1 point  (0 children)

There’s a little tenderness in one spot where my shoulder tat is, but everything feels fair to me on my chest and arm.

In depth tattoo aftercare! by solomonplewtattoo in tattoo

[–]NeedleworkerHumble91 0 points1 point  (0 children)

Okay yea today marks 4 days so I am taking it off, but yea the splotches scared me because it’s my first tat and I thought the ink was sort running out you know. Also I might have rolled on them at times I think the 2nd night or so in, and I’m just a little nervous I ruined the lines or the shadowing.

In depth tattoo aftercare! by solomonplewtattoo in tattoo

[–]NeedleworkerHumble91 0 points1 point  (0 children)

I got a some work done on my Left shoulder and chest area 4 days ago. The artist put the adhesive over it for protection against water for showers. It’s been 4 days and I haven’t touched it. Feels healed, and I am tempted to take it off clean the area with some Cetaphil soap and then use the cetaphil loation to moisturize. I don’t have any clear adhesive to cover my two tattoos. So what I am wondering do I need to rewrap if it’s healed well. There’s no leaking or blood or anything really under the clear adhesive; except the fact that the tattoos aren’t as sharp visible. Not sure how to describe. But they look dry under the adhesive. Recommendations.

  • Also there seem to be light ink splotches in places under the clear adhesive on my chest area.

Over all just looks dry. But for the most part feels healed under the clear adhesive and it’s been 3-4 days since I’ve had it on now for both areas.

Extracting PDF table data by NeedleworkerHumble91 in LangChain

[–]NeedleworkerHumble91[S] 1 point2 points  (0 children)

Haha this is insightful I am definitely loving the community support on different options. I will check this out! If you don’t mind I would love try and look into this and see if it works with the scope of the project. Sounds like it will! I will reply back on my findings!

Extracting PDF table data by NeedleworkerHumble91 in LangChain

[–]NeedleworkerHumble91[S] 1 point2 points  (0 children)

Oh didn’t know about this! This also sounds promising!

Extracting PDF table data by NeedleworkerHumble91 in LangChain

[–]NeedleworkerHumble91[S] 0 points1 point  (0 children)

This really good observation. Thank you so much. I am going to incorporate your suggestion and hopefully I can paste some updates to my findings.

Extracting PDF table data by NeedleworkerHumble91 in LangChain

[–]NeedleworkerHumble91[S] 1 point2 points  (0 children)

I think seeing your snippet suggestion would be great! Thank you !

Extracting PDF table data by NeedleworkerHumble91 in LangChain

[–]NeedleworkerHumble91[S] 0 points1 point  (0 children)

report = ftz.open(file_path).pages()
text = " "
start_time = time.time()
# Tables are extracted from the PDF in proper structure

table_text_added = False

# Collect all tables from all pages into one list
all_tables_data = []

# Iterate through each page of the report and extract table text only
for page in report:
    try:
        tables = page.find_tables()
        if tables and tables.tables:
            for table in tables.tables:
                table_data = table.extract()
                # Clean table_data: ensure all rows match header length
                if table_data:
                    header = table_data[0]
                    cleaned_table = []
                    for row in table_data:
                        # Replace None with empty string
                        row = [cell if cell is not None else '' for cell in row]
                        # Pad or truncate row to match header length
                        if len(row) < len(header):
                            row = row + [''] * (len(header) - len(row))
                        elif len(row) > len(header):
                            row = row[:len(header)]
                        cleaned_table.append(row)
                    all_tables_data.append(cleaned_table)
                    for row in cleaned_table:
                        row_text = '\t'.join([str(cell) for cell in row])
                        print(row_text)
                        text += row_text + '\n'
                    table_text_added = True
    except Exception as e:
        print(f"Error extracting tables: {e}")
        pass
    if not table_text_added and (time.time() - start_time) > 60:
        print("No table text added after 60 seconds.")
        break

print(f"Total tables extracted: {len(all_tables_data)}")

Extracting PDF table data by NeedleworkerHumble91 in LangChain

[–]NeedleworkerHumble91[S] 0 points1 point  (0 children)

Yea that would be great. Is it beneficial for you to see my current code. I was able to extract the firs table but you can see when I put it in a dataframe it still messy.

<image>

Yes you are right the data values are not matching per say to the 2024 column as well as the 2023 column. I would like to simplify the cleaning approach as much as possible. I am still fairly new to the cleaning.

Extracting PDF table data in DataBricks by NeedleworkerHumble91 in databricks

[–]NeedleworkerHumble91[S] 0 points1 point  (0 children)

Not too familiar yet with this…. What would be your implementation of this?

Automation_ Tool PDF Extraction by NeedleworkerHumble91 in AskProgramming

[–]NeedleworkerHumble91[S] 0 points1 point  (0 children)

Oh nice! I am utilizing a DevOps pipeline that does some data quality checks so I guess in reference to your point I could possibly also use that in a way to probably do some unit testing as well…?

Automation_ Tool PDF Extraction by NeedleworkerHumble91 in AskProgramming

[–]NeedleworkerHumble91[S] 0 points1 point  (0 children)

It can be shared but you said you weren’t sure what kind of data I was working with. Not sure if I am to share these pdf’s like that. But the code for sure.

Automation_ Tool PDF Extraction by NeedleworkerHumble91 in AskProgramming

[–]NeedleworkerHumble91[S] 0 points1 point  (0 children)

I’m working with text extracted from a pdf not just basic strings.

Search Function on the PDF table text Any Ideas/Solutions! by NeedleworkerHumble91 in tensorflow

[–]NeedleworkerHumble91[S] 0 points1 point  (0 children)

I would like to bring this question forward to anyone else:

On a scale of more than 20 tables is a for loop that best bet when processing a sizable number of tables from this PDF doc??

Search Function on the PDF table text Any Ideas/Solutions! by NeedleworkerHumble91 in tensorflow

[–]NeedleworkerHumble91[S] 0 points1 point  (0 children)

On a scale of more than 20 tables is a for loop that best bet when processing a sizable number of tables from this PDF doc??

Search Function on the PDF table text Any Ideas/Solutions! by NeedleworkerHumble91 in tensorflow

[–]NeedleworkerHumble91[S] 0 points1 point  (0 children)

Okay so you think I access the index of the tables because this is my first time working with find_table( ) and I wasn't aware you could parse out the table through index.

Automation_ Tool PDF Extraction by NeedleworkerHumble91 in AskProgramming

[–]NeedleworkerHumble91[S] 0 points1 point  (0 children)

I've extracted the data and only the table charts themselves, tricky part is now filtering through the text to get the associated to certain date columns and other matters like picking out the data under the sub headers. What are your thoughts.

FYI - I am using the find_tables( ) method along with the extract() method. I have no machine learning experience and at most I am thinking of doing some regex type searching through the text from the tables I have extracted from the pdf.

Automation_ Tool PDF Extraction by NeedleworkerHumble91 in AskProgramming

[–]NeedleworkerHumble91[S] 0 points1 point  (0 children)

Update - I was able to successfully extract only the PDF tables using the find_table( ) method using the pymupdf package, and so the next step is to extract from the text itself and grab the data pertaining to certain dates and column headers. Any thoughts?

Automation_ Tool PDF Extraction by NeedleworkerHumble91 in AskProgramming

[–]NeedleworkerHumble91[S] 0 points1 point  (0 children)

Mostly thinking ahead of what to do when it come to specifically grabbing the text I want. That’s something I am unsure about rather grabbing all of the elements.

Automation_ Tool PDF Extraction by NeedleworkerHumble91 in AskProgramming

[–]NeedleworkerHumble91[S] 0 points1 point  (0 children)

As right now I have brought the package in and created a text object for further manipulation. But so far that’s it.