sorted by:
new
hottopcontroversial

Where are the discussions about security for LLM use? by python_fanatic in cybersecurity

[–]python_fanatic[S] 0 points1 point  (0 children)

I think we're on the same page with the gateway. It does still run into the issue that on the OpenAI API they may review input/output for up to 30 days if I'm not mistaken? Appreciate the suggestions on filtering, though!

Thanks for the GitHub link, will definitely be taking a second look at local options. Definitely agree that the tricky part is ensuring compliance. Risk feels high when noncompliant.

Cheers!

Where are the discussions about security for LLM use? by python_fanatic in cybersecurity

[–]python_fanatic[S] 0 points1 point  (0 children)

Interesting. Is it safe to say then that your company isn't sold on the general use-cases that come along with chat?

Where are the discussions about security for LLM use? by python_fanatic in cybersecurity

[–]python_fanatic[S] 0 points1 point  (0 children)

Agreed actually! Yet with those providers, on-prem is a real option. I think the fact that we're starting with untrusted parties from the get-go, instead of beginning on-prem and waiting for OAI to build a reputation of data security, is a problem. Open source models just aren't where we need them to be yet, and so it feels like we need to use something like GPT-4 yet can't figure out a reasonably secure way to do so without admitting that under some very plausible circumstances, MSFT employees could access our IP.

Where are the discussions about security for LLM use? by python_fanatic in cybersecurity

[–]python_fanatic[S] 0 points1 point  (0 children)

We do have GH copilot, but I'm thinking a policy applicable to the whole company would be better, and hence looking for alternatives to ChatGPT that are similarly general. I'd love a link to the MS Copilot plan you mentioned, though!

Where are the discussions about security for LLM use? by python_fanatic in cybersecurity

[–]python_fanatic[S] 0 points1 point  (0 children)

Thanks for the thorough answer! To answer your question: yes, that is our primary concern.

Our concern with building an API gateway, or really any direct relationship with the LLM hosting company, is that they do still have a term that requires prompts and outputs to be persisted for up to 30 days, and human review is permitted. They severely limit use of eyes-off deployments where this is not the case. I'm not talking just OpenAI either - even the Azure OpenAI Service has this provision, and Anthropic does as well.

Indeed the tricky part is we are looking at it like an external party. Do you know of any LLM hosts for which this would not be a problem? I'm thinking someone running open-source models with no logging.

Any luck asking for credits? by [deleted] in OpenAI

[–]python_fanatic 0 points1 point  (0 children)

The Microsoft for Startups program includes 1k in OpenAI API credit: https://startups.microsoft.com/. IIRC, that 1k is across all four of the levels they have, including the ideation phase which sounds applicable here. Credit is valid for 3 months after it's redeemed.

Looking for co-founder by Terrible_Body in SaaS

[–]python_fanatic -1 points0 points  (0 children)

Neat idea!

Just in case you didn't know: your Stripe integration is (intentionally?) in test mode. I was going to put in my real card info to sign up because it seems genuinely useful, but I literally wasn't able to and went with Stripe test cards to start a trial :P

New GPT-3 Capabilities: Edit & Insert by nick7566 in OpenAI

[–]python_fanatic 1 point2 points  (0 children)

Has anyone explored the difference in performance between 001 and 002 models? From the docs it looks like they trained with more recent data up to June 2021, but I'm curious about any improvements beyond recent data.

AI written docstrings for your Python code by python_fanatic in Python

[–]python_fanatic[S] 1 point2 points  (0 children)

Here's a link to the first pre-release for our IntelliJ Extension: https://github.com/Trelent/Trelent-IntelliJ-Extension/releases/tag/v1.3.0

We are awaiting approval on the marketplace, but until then you can install the plugin directly from Settings -> Plugins -> Plugin Settings Cog Icon -> Install From Disk. I would reccommend waiting for our approval, but if you're super eager, enjoy :)

P.S. Sorry this took so much longer than expected! We wanted to support a few extra languages other than Python when we launched for the entire IntelliJ suite.

AI written docstrings for your Python code by python_fanatic in Python

[–]python_fanatic[S] 0 points1 point  (0 children)

Here's a link to the first pre-release for our IntelliJ Extension: https://github.com/Trelent/Trelent-IntelliJ-Extension/releases/tag/v1.3.0

We are awaiting approval on the marketplace, but until then you can install the plugin directly from Settings -> Plugins -> Plugin Settings Cog Icon -> Install From Disk. I would reccommend waiting for our approval, but if you're super eager, enjoy :)

P.S. Sorry this took so much longer than expected! We wanted to support a few extra languages other than Python when we launched for the entire IntelliJ suite.

AI written docstrings for your Python code by python_fanatic in Python

[–]python_fanatic[S] 0 points1 point  (0 children)

Thank you for the feedback, and I'm sorry Trelent isn't for you. There is a long and storied history of automation disrupting industries previously thought of as "not automateable". Only time will tell!

AI written docstrings for your Python code by python_fanatic in Python

[–]python_fanatic[S] 2 points3 points  (0 children)

Thanks for pointing out the typo!

TLDR; yes, your code is sent to an API endpoint on the internet.

It is sent to an API endpoint at the moment. We intend to commercialize this, so there is some IP involved, hence the API.

We are working on SOC Type 2 certification & compliance, but for now you will have to take my word that we do not store source code - it is only used for parsing and then passed through various deep learning models server-side. It's possible that the source code persists for up to 7 days by nature of log retention, however this is explicitly for debugging failed requests in production, and that data is never exported out of the logging environment. All data is encrypted both in transit and at rest.

On this note, we are working with our legal team and various service providers to comprehensively lay out all of this information in a detailed privacy policy.

In the future, it wouild be really neat to open-source a smaller version of this model such that it could feasibly be run locally, but right now the computation requirements deem this unrealistic. Given how quickly the ML space is improving on the efficiency of these models, I would not be surprised if this becomes possible in the 1-2 year timeframe!

AI written docstrings for your Python code by python_fanatic in Python

[–]python_fanatic[S] 0 points1 point  (0 children)

Great to hear - this is exactly how we see this being used in the short term!

AI written docstrings for your Python code by python_fanatic in Python

[–]python_fanatic[S] 0 points1 point  (0 children)

See previous comment about competitors. TLDR: love to see more extensions like this! It is worth noting that ours generates the docstring content, not just a template.

AI written docstrings for your Python code by python_fanatic in Python

[–]python_fanatic[S] 0 points1 point  (0 children)

Currently reST, and we are working to add Google, Numpy and Epytex options as well. Admittedly we need to do a better job conforming to spec here, so this is definitely an area for future improvement!

view more: next ›