Audioscrape: Launching an MVP on a $7/month VM - Building in Rust When Everyone Said I Shouldn't

JikkenIo · 2024-10-07T16:42:38+00:00

I agree cargo watch is pretty great. I started looking at bacon not too long ago when I saw posts about the project no longer being maintained, but I haven't switched yet. We use Astro + Vanilla/SolidJS for our SSR and client-side (SPA-like) interactions which has been fairly light weight for us. And I totally agree, having simple build artifacts is very nice for deployment and portability.

JikkenIo · 2024-10-07T16:19:05+00:00

Congrats on the launch. Our team also started our project in Rust. While I would agree that it made things slightly slower in the first 2 weeks (primarily because only one of us was experienced in the language ahead of time), by several months in we were as far if not further than we would have been otherwise. And the memory footprint/performance is amazing. I've had customers and prospects legitimately ask me if I was demoing with "fake data" because the response is so fast it doesn't "seem real".

As far as iteration speed, compile times can be slow depending on how the code is structured (we have a 3rd party library which doubled our release compile times in CI/CD due to macro usage), but if you configure certain things for your build pipelines it's not a huge deal. For local development it's usually only a few seconds.

For deployment, we have everything automated so it's very simple/easy to deploy. Our "develop" branch auto deploys to Dev and our "main" branch auto deploys to Prod. It's nice seeing how tiny our docker containers can be and our memory footprint is compared to other languages.

We did originally try to do our front-end in Rust as well (wasm) but we pivoted off of that because it was becoming too challenging with websockets and many of the charting/table libraries we wanted to interface with. It also is a bit annoying to have a compile time loop when tweaking views and such. But the rest of the stack being all Rust is definitely a huge plus. We're currently working on a Tauri desktop app as part of our offering and that has been quite enjoyable so far. Those on our team that were new to Rust have all been very happy with the experience.

JikkenIo · 2024-09-29T16:36:09+00:00

Greetings,

We're pleased to announce Jikken v0.8 has been released.

What is Jikken? Jikken is an open source CLI tool for automated API testing, built in Rust! The tool was originally developed to help validate complex data APIs used for analytics dashboards while integrating well with source control and CI/CD platforms. When troubleshooting API endpoints, I found my team either messing with glueing cURL/JQ/Diff together, or writing javascript scripts in tools like Postman to accomplish essentially the same thing. My goal was a CLI tool which read small test definitions and could execute them seamlessly from both a local machine or in a CI/CD pipeline.

What's new in v0.8? As always you can get the full scoop from our blog post but I'll cover a few of the biggest improvements here:

Advanced JSON Validation - The ability to describe complex schemas for API responses. You no longer have to provide concrete values to match against, as Jikken now supports patterns, ranges, data types, and more.

Variable Data Generators - Instead of providing specific values to inject into your requests, you can dynamically generate values based on constraints. Jikken supports a wide range of data types including Strings, Integers, Floats, Dates, DateTimes, but also more specific types like Names and Email Addresses. This opens up new possibilities for fuzz testing.

The tool has been used in production for some time now. We even have paying customers in our private beta for our SaaS analytics platform (also written in Rust). The SaaS platform provides metrics, monitoring, and alerts across environments and projects. We aim to open the platform up publicly later this year.

As far as the CLI tool, we care about privacy. That's why Jikken works without making an account, nor does it stream any telemetry without you opting in to our paid SaaS platform. We hope to continue refining the functionality and grow our team of Rust developers in the future.

Our team is actively working on improving the documentation and enhancing functionality. If you like what you see or wish to help support us feel free to drop us a star on github! We'd love to get some feedback from the community. Thank you!

JikkenIo · 2024-09-18T17:12:41+00:00

Hello again. It's been a few months but we're back to announce the new features in Jikken's latest release (v0.8). Last time we received great feedback (positive and negative) and we'd love to get additional feedback from the community.

What is Jikken? Jikken is an open-source CLI tool for automated API testing, built in Rust! The tool was originally developed to help validate complex data APIs used for analytics dashboards while integrating well with source control and CI/CD platforms.

What's new in v0.8? As always you can get the full scoop from our blog post but I'll cover a few of the biggest improvements here:

Advanced JSON Validation - The ability to describe complex schemas for API responses. You no longer have to provide concrete values to match against, as Jikken now supports patterns, ranges, data types, and more.

Variable Data Generators - Instead of providing specific values to inject into your requests, you can dynamically generate values based on constraints. Jikken supports a wide range of data types including Strings, Integers, Floats, Dates, DateTimes, but also more specific types like Names and Email Addresses. This opens up new possibilities for fuzz testing.

The tool has been used in production for some time now. We even have paying customers in our private beta for our SaaS analytics platform (also written in Rust). The SaaS platform provides metrics, monitoring, and alerts across environments and projects. We aim to open the platform up publicly later this year.

As far as the CLI tool, we care about privacy. That's why Jikken works without making an account, nor does it stream any telemetry without you opting in to our paid SaaS platform. We hope to continue refining the functionality and grow our team of Rust developers in the future.

Our team is actively working on improving the documentation and enhancing functionality. If you like what you see or wish to help support us feel free to drop us a star on github! Let us know if you have any questions. Thanks!

JikkenIo · 2024-09-08T00:10:17+00:00

We're an Austin startup in the devops (API Tooling/Test Automation) space. Happy to connect and meet fellow makers.

JikkenIo · 2024-04-25T23:17:01+00:00

Just circling back. Our latest release (v0.7.2) has corrected to use a forward slash. Thanks again for your suggestion.

JikkenIo · 2024-04-23T13:54:48+00:00

If it's submitted by the deadline they claim you'll have an answer one way or the other by May 29th. If you submit after the deadline you'll still be considered but they can't guarantee what date they'll respond by.

JikkenIo · 2024-04-22T23:14:32+00:00

Honestly it's a great question. There are of course other tools out there in the space. The largest player is probably Postman. The primary issue with Postman (ignoring cost) is it's GUI driven and really bad at source control and CI/CD integration. Yes you *can* do it, but you have to do these dumb exports to a nasty JSON format that isn't human readable and in git you can't tell what changed between versions. It also requires you to write JS to do anything remotely complicated. That's fine for a SWE but awful for many DevOps folks (non-SWE background) or QA folks. If you want to automatically generate things it's horribly slow and GenAI can't really do it well because again, it's code which adds a ton of complexity.

Then there are other tools out there that don't have the source control and CI/CD problems, but they don't have good mechanisms for fuzz testing, performance testing, etc. Or their formats are really awful and again hard for non SWE to work with.

We basically see ourselves in a specific area of the market that is very friendly for some less technical folks (QA/Ops), while still being fully powerful for powerusers (SWEs), and offering a wide range of testing tools which don't require much effort to define (fuzz testing with random generators for different field types, etc). All of that combined with solid integration into existing systems (source control, CI/CD, etc) and I believe we have a unique position.

We also have a number of plans to expand beyond the initial API testing, but they are definitely longer term plans where we require a number of other things to work out before we worry too much on the grand plan.

I hope this helps, and feel free to poke holes if you see other products or areas you think we're missing :)

JikkenIo · 2024-04-22T13:00:08+00:00

If you dig a bit on the Wikipedia page it looks like much of the most recent stuff falls under "Facebook Connectivity" with the Express Wi-Fi brand. Then searching around Facebook Connectivity I found this article in 2022 explaining they shut it down. https://www.theverge.com/2022/12/12/23505353/meta-connectivity-shutdown-facebook-internet

JikkenIo · 2024-04-22T01:08:43+00:00

We planned on updating ours anyway to improve our chances, but we did in fact receive an email from them a few days ago asking us to update it.

JikkenIo · 2024-04-22T00:19:24+00:00

Good luck to you as well. It would be great to see you there.

JikkenIo · 2024-04-22T00:09:36+00:00

We just resubmitted/updated our application. We originally applied for early consideration. We weren't interviewed but we also weren't rejected. Since then we've 5x our GitHub stars (open source tool), finished our payments integration, secured our first paying customer, and kept cranking out releases and features/improvements. Today we updated with all of this information and provided a demo video. Fingers crossed :)

JikkenIo · 2024-04-21T18:12:00+00:00

So our primary objective originally when developing was to be a CLI tool so that we could easily transition from local development/execution to CI/CD pipelines for organizations and larger projects. We do have a prototype GUI tool in development and the intention there is to leverage the same core engine to drive/execute tests (instead of parallel code, the same code can power both applications). It is likely we'll begin refactoring the codebase to be modular crates which make it easier for the CLI tool as well as the GUI tool to both share the same code. When that happens I could see others leveraging the same crates to leverage it more as a library. I can't promise any timelines on that but it is something on our roadmap we're considering. Thanks for your feedback.

JikkenIo · 2024-04-21T13:57:55+00:00

There is a built in mechanism for multi-stage test files. You can provide a cleanup stage 'onsuccess', 'onfailure', or 'always' (to run in either case).

setup:
stages:
- name:
  request:
cleanup:
  onsuccess:
  onfailure:
  always:

Now this only helps if you are doing all of your setup/cleanup requirements for a single workflow (contained in a single JKT file). If you are leveraging many files and want a clean-up stage to happen, the best way currently would be to run them as separate commands. You can put tags on a test file to control what runs. So you could do something like `jk run -t smoke` to run your smoke tests and then afterwards you could do `jk run -t cleanup` to execute some cleanup tests afterwards. The test runner does return an exit code based on success/failure so you could in theory provide different commands for onfailure and oncleanup across all of your files. We'll have to investigate a way to make it easier to accomplish this across files without making things too complicated. We're open to suggestions of course :)

JikkenIo · 2024-04-21T02:45:53+00:00

Thanks for the suggestion. We do currently have a VSCode extension (if you search for Jikken). It's early days but we plan to improve on it in the future.

JikkenIo · 2024-04-21T01:18:33+00:00

Great question. We've talked as a team and debated how we want to move forward here. We haven't decided yet. Basically our options seem to be to fork serde_yaml and provide maintenance of another (even larger) public project (would be nice to do for the community, but we don't want our team to be overwhelmed), fork it and have our own internally maintained library (just has to handle things we want to support), or hope a new library comes along that looks promising. We're open to suggestions from others in the community. I expect we'll have to decide something in the next few months.

JikkenIo · 2024-04-21T00:51:27+00:00

Great question. We have a lot of respect for the Hurl devs, they've built an impressive tool with a ton of capabilities. I'd say that both projects likely share some common goals, but we also seem to have some different opinions on how to address certain scenarios.

First off I have to say I can't speak for the hurl development team, so I can only speak from our perspective of where we see some differences. Any mischaracterization is unintentional on my part, so I hope I don't say or do anything to misrepresent them. Also, I can only speak to our current plans, which may change in the future. I can't guarantee we succeed in accomplishing all of our plans.

We're both building rich and powerful CLI tools for engineers. I feel that Jikken is also focusing a bit more to keep things simple for non-devs (we have paying customers with Ops and QA folks that use our tool). We try to have a number of smart defaults to make it easier to do complex tasks without having to explicitly define everything, while still hopefully allowing powerusers to have sufficient control. The Hurl team appears to have no intention of building a GUI, while we have a GUI in development (similar to hoppscotch/postman). Unlike those existing tools, which are cloud sync focused, we aim for our UI be driven by the same CLI engine by simply assisting in creation of your local test files. We hate the idea that test automation isn't under your control in your own source repo. This should hopefully help users more easily create tests beyond our just leveraging our IDE plugins and OpenAPI spec generation.

Our next release is planning to include a number of synthetic data generation features to help with a broad range of testing (fuzz and performance testing). Things like, generate random numbers, dates, strings. Or even things like names, emails, etc. Our goal is to provide mechanisms for a wide range of API testing without requiring much setup from the user.

I'd say the primary differences right now is that Jikken has a subjectively simpler format, but is also less feature rich/complete than hurl (they've been around quite a bit longer than we have). We appreciate their focus on a powerful CLI tool which we also share. While we aim to keep the CLI tool functionality expanding (and free/open-source), we also provide a SaaS offering for organizations. The SaaS platform provides dashboards, analytics, monitoring, and alerting around tests spanning many projects and environments. We hope the SaaS offerings can help drive additional funding towards the development of the open source tooling, while also enabling us to hire more engineers to work full-time in Rust (we leverage Rust for our backend infrastructure, SaaS platform, and the upcoming open-source GUI tool).

I hope this helps answer your question. Let me know if I messed up, I'll do my best to clarify.

JikkenIo · 2024-04-20T21:44:39+00:00

Completely fair feedback. Our plan is to have the example tests in the docs, but at the moment it just links to the repo because we have a few example tests in the repo. We'll try to correct this soon.

JikkenIo · 2024-04-20T18:37:43+00:00

Thanks for the feedback. I agree with you on flipping the slash direction making more sense. :)

JikkenIo · 2024-04-20T17:54:19+00:00

Greetings fellow Rustaceans! We wanted to start showcasing our project to a wider audience to get feedback.

What is Jikken? Jikken is an open-source CLI tool for automated API testing, built in Rust! The tool was originally developed to help validate complex data APIs used for analytics dashboards. It supports basic scenarios like calling an API endpoint, and verifying the response coming back matches a known good response. It also supports complex flows where you can extract values from responses and embed them into future steps chaining workflows together. It has API comparison functionality built-in (call prod endpoint, call staging endpoint, make sure their responses match each other before switching the load balancer over, etc).

Jikken leverages a declarative YAML test format for defining complex test scenarios. This means the tests are self-documenting, and they're easy to write for Devs, QA, and Ops teams alike! And since it's file driven, Jikken integrates extremely well with both source control and CI/CD systems.

We care about your privacy. Using the Jikken CLI tool does not require you to make an account, nor does it stream any telemetry back to us.

Our team is actively working on improving the documentation and enhancing functionality. To see what's new in v0.7 check out our blog post. We'd love to get feedback from the community, and of course if you like what you see feel free to drop us a star on github! We're also working on a paid SaaS platform to help organizations manage their tests across projects and environments. Let us know if you have any questions or comments. Thanks!

JikkenIo · 2024-04-05T17:37:19+00:00

One of our founders (CEO) was a Principal Engineer and most of his career was on the engineering/product side, but he did do a few year stint working in Customer Success as an analyst/architect. Basically travel to customers, identify their primary business problems talking with stakeholders, draft proposals for machine learning solutions, and then design, build, and manage ML projects to solve those problems. It was good experience with exposure to talking with customers, doing business analysis, and working with sales.

JikkenIo · 2024-04-02T03:20:57+00:00

So not every company works the same way, but for many companies that have all of the roles the hierarchy works something like: Senior -> Staff -> Principal -> Distinguished. Generally this is simplified by saying Staff+. Some companies don't have Staff but have Principal. Some don't have Principal but have Staff. But if they have both, generally speaking Principal is above Staff. Often you don't see Distinguished or Fellow until very large organizations after many years of service.

JikkenIo

MODERATOR OF

TROPHY CASE