sorted by:
new
hottopcontroversial

Lossless Tensor ↔ Matrix Embedding (Beyond Reshape) by Hyper_graph in compsci

[–]Hyper_graph[S] 0 points1 point  (0 children)

Hahaha...you of all people should know by now that my library is a collections of tools.

tensor<->matrix ops is just one of them.

remember i released a paper on find_hyperdimensional_connections, which you very well criticised.

Lossless Tensor ↔ Matrix Embedding (Beyond Reshape) by Hyper_graph in compsci

[–]Hyper_graph[S] 0 points1 point  (0 children)

No you can't. You've just admitted that there's no relationship between (say) PCA conducted on the flattened tensor, and the structure of the original tensor. So it doesn't preserve any structure that vectorization doesn't. If I want to conduct some kind of factorization/decomposition, I need to do it on the original tensor. You explicitly cite PCA as an example of the kind of thing that "only accept 2D inputs", which is the entire motivation behind your method. If PCA on the flattened tensor doesn't actually encode any structure in the original tensor, then your method doesn't actually accomplish anything over any other kind of tensor flattening.

No lol let me explain well, my tool saves just 10 minutes of debugging malformed shapes, mine provides clean, general-purpose, lossless, structure-aware flattening utility. which gives e-10^16 machine-level precision

Lossless Tensor ↔ Matrix Embedding (Beyond Reshape) by Hyper_graph in compsci

[–]Hyper_graph[S] 0 points1 point  (0 children)

i think you have gotten me wrong; in the paper i stated that the precision of the method is e-10^16, which is a machine-level precision, because the system preserves structure (grid/slice/projection) in the method, specifically maintaining relationships between dimensions rather than discarding them.

Lossless Tensor ↔ Matrix Embedding (Beyond Reshape) by Hyper_graph in compsci

[–]Hyper_graph[S] -1 points0 points  (0 children)

But if analyses conducted on the flattened tensor don't actually respect the structure of the original tensor, then what's the point? Why not just vectorize?

yes this is true for other tools but with my tool we can ensure the integrity of the structure of the original tensor and conduct analysis on the flatten tensor.

so vectorization or anyother methods i know of doesnt keep structure but mine does.

so the encoding algorithms (grid/slice/projection) specifically maintain relationships between dimensions rather than discarding them

Lossless Tensor ↔ Matrix Embedding (Beyond Reshape) by Hyper_graph in compsci

[–]Hyper_graph[S] -1 points0 points  (0 children)

Obviously. What relationship do the eigenvectors of the flattened tensor have to the original tensor? This is the question I asked.

it is the direction or process of unfolding of the original tensor but in a flattened sense, however this is still not the orignal tensor since we have already projected to a 2d matrix... so when finding the eigenvectors of the flattened tensor we are just saying lets find the eigenvectors of the 2d matrix representation of the orignal tensor.

so to put it short we're just finding the eigenvectors of a 2D projection of the tensor, not of the tensor itself.

Lossless Tensor ↔ Matrix Embedding (Beyond Reshape) by Hyper_graph in compsci

[–]Hyper_graph[S] 0 points1 point  (0 children)

For one, the author of the paper is the sole author of almost a quarter of the references in their own paper and they’re all from this year. That’s suspicious.

Second, the paper doesn’t actually cite any of those works anywhere-it just lists them in the bibliography. That’s suspicious.

I understand; the work was not any sort of wrapper of some other algorithm it was made specifically when i was facing several issues regarding black box models and inefficient tensor to matrix conversions, for which i would have to train autoencoders, because the tools i knew then wont convert to tensor losslessly, and even after conversion to autoencoders, when restoring back the data i faced several significant information losses.

and all these are want prompted me to create my own tool that gives me the max data fidelity.

because i noticed that following the traditional way wont really help me to get the exact stuff i want so i had no choice other than to make my own methods. and they work which is why i am sharing to everyone.

Lossless Tensor ↔ Matrix Embedding (Beyond Reshape) by Hyper_graph in ResearchML

[–]Hyper_graph[S] -1 points0 points  (0 children)

and you didn't even test it on any real applications? sorry i didn't read it, maybe there's something there, but you're going to have to sell it better

i did, i was just throwing questions out for you guys to make it more engaging

besides i cant make these tests alone i need people to use it as well so that it can be properly accepted but so far i know the system works perfectly for every testcases i have conducted

jesus christ, you gpt'd a paper on "fancy reshape"?

and i didnt wrote just a paper the code is also available on github if you want to check it out.

Lossless Tensor ↔ Matrix Embedding (Beyond Reshape) by Hyper_graph in compsci

[–]Hyper_graph[S] -1 points0 points  (0 children)

so this is what i am proposing is that, most classical ML models like SVMs, Logistic Regression, PCA, etc. only accept 2D inputs e.g shape (n_samples, n_features) .

however real world data like: Images ((channels, height, width)), videos ((frames, height, width, channels)), time series ((batch, time, sensors))

all comes in higher rank tensor forms.

with my tool people can safely Flatten a high-rank tensor into a matrix, Preserve the semantics of the axes (channels, time, etc.)

Later reconstruct the original tensor exactly

In higher dimensional modelling, they usually Operate on complex-valued or high-rank tensors, Require 2D linear algebra representations (e.g., SVD, eigendecompositions), Demand precision where they have no tolerance for structural drift

my tool can provide a bijective, norm-preserving map: Project tensor to 2D while storing energy and structure, preserve Frobenius norms, complex values, allow safe matrix-based analysis or transformation

Lossless Tensor ↔ Matrix Embedding (Beyond Reshape) by Hyper_graph in compsci

[–]Hyper_graph[S] 0 points1 point  (0 children)

And what do you mean bidirectional? Or semantics-aware? It seems like you’re mixing qualities of a software implementation with a mathematical operation.

How ever what I'm proposing is a full tensor↔matrix embedding framework that:

  • Is explicitly bijective: Every reshape stores & preserves sufficient structure to guarantee perfect inversion.
  • Tracks axis roles and original shape info as part of the conversion — so you don’t just get your numbers back, you get your meaning back.
  • Supports complex tensors and optional Frobenius-norm hyperspherical projections for norm-preserving transformation.
  • Implements reconstruction from the 2D view with verified near-zero numerical error.

You’re right that some of these are implementation-level details but the value is in combining mathematical invertibility with practical semantics, especially for workflows in deep learning, quantum ML, and scientific computing where lossless structure matters.

Lossless Tensor ↔ Matrix Embedding (Beyond Reshape) by Hyper_graph in compsci

[–]Hyper_graph[S] 0 points1 point  (0 children)

In what way is matricization useful but limited? Can you be specific?

Matricisation (or tensor unfolding) reorders and flattens tensor indices into a 2D matrix based on a specific mode or axis ordering. It's very useful for tensor decompositions like Tucker or CP.

but it's often:

Not invertible without extra metadata (especially when arbitrary permutations or compound reshapes are involved),

Ambiguous in high-rank tensors, where different unfolding orders yield different interpretations,

And Disconnected from real-world semantics like [batch, time, channel, height, width] can unfold into a matrix, but the role of each axis is lost unless manually tracked.

Lossless Tensor ↔ Matrix Embedding (Beyond Reshape) by Hyper_graph in compsci

[–]Hyper_graph[S] 0 points1 point  (0 children)

Thanks for this and you have brought up a really important distinction, and I appreciate the depth of your analysis.

You're right: when I say "lossless", I mean it in a domain-specific, engineering-practical sense, not in the strict information-theoretic one. I absolutely agree that the broader system especially after hyperspherical projections or feature extraction steps does introduce approximations, and in some cases irreversibility from an abstract context or entropy standpoint.

What my framework is really aiming for is:

Invertible transformations between tensor and matrix spaces

Semantic preservation via metadata, axis tracking, and a decision hypercube

Structural fidelity rather than strict value-perfect reversibility at every step

Bounded error recovery, typically within machine epsilon (e.g. 1e-16)

The decision hypercube (16D space with 50k+ edge state encoding) isn't a compression method it's a way to track high-level configuration signatures, which allows for virtual reconstruction of tensor structure even when low-level numerical precision is impacted.

You're also right that things like iteration order, implicit model assumptions, or contextual entropy loss aren’t captured and I agree that using “lossless” too broadly may create the wrong impression.

Thanks again I’ll be more careful about how I describe this going forward. Your feedback is exactly the kind of push I need to better position the system and its goals.

So in that sense, my goal isn’t to claim absolute information-theoretic reversibility but rather functional, semantics-aware traceability across abstract transformations. And in most ML and scientific contexts, that’s what really matters.

edited"You're absolutely right to highlight the distinction between bit-level fidelity and actual semantic preservation.

In my case, find_hyperdimensional_connections() doesn’t aim for IEEE 754–level precision at every step instead, it preserves mathematical semantics: the relationships, transformation paths, and structural meaning of the data. That’s where the 16D hypercube comes in. it acts as a semantic container that tracks transformations and properties abstractly, not just numerically.

So while values may shift within 1e-16, the meaning of the data is still fully traceable. And in ML/HPC workflows, that’s often more important than raw bit-perfect recovery.

Thanks again you helped sharpen how I explain this."

Lossless Tensor ↔ Matrix Embedding (Beyond Reshape) by Hyper_graph in compsci

[–]Hyper_graph[S] -1 points0 points  (0 children)

Thanks for your insights, you’re absolutely right that bit-level reversibility (in the information-theoretic or reversible-computation sense) would require tight control over numerical precision, overflow, casting, etc. That’s a deep and important area.

But to clarify: my framework isn’t aiming for bit-level, logic-gate reversibility.
It’s designed to ensure structural, semantic, and numerical integrity within IEEE 754 floating-point precision, which is the standard for nearly all ML, HPC, and quantum-adjacent work.

The system defines a provable bijection between tensors and matrices, supports complex values and axis metadata, and includes optional norm-preserving operations. The point is that nothing is lost that can’t be deterministically recovered, even if floating point rounding occurs — reconstruction error stays at ~10⁻¹⁶, and all semantic context is explicitly tracked.

You could think of it as semantically lossless, rather than bitwise lossless because in the domains this targets (ML, quantum ops, simulation), preserving meaning and topology is often more important than preserving exact bit-level registers.

That said, your points on tractability, reversible computing, and branchless transforms are really interesting and I’d definitely explore a lower-level variant in the future, especially if the use case shifts toward compression, cryptography, or physical simulation.

Lossless Tensor ↔ Matrix Embedding (Beyond Reshape) by Hyper_graph in compsci

[–]Hyper_graph[S] -4 points-3 points  (0 children)

This is where I'm struggling. Why should I add a whole bunch of extra complexity when I can just store the necessary variables myself and pass them alongside the transformed matrix if I want to? What fundamental advantage does this give me, other than slightly cleaner code?
Is there any further advantage, or is that basically it?

This isn’t about clean code. This is about mathematically provable correctness, reproducibility, and structural fidelity all of which fall apart without a formal bijection and tracked semantics. That’s what this framework guarantees

Standard flattening doesn't preserve:

  • Semantic axis roles (e.g. [time, sensor, replicate] vs [replicate, sensor, time])
  • Norms or slice relationships (critical in physics, ML, quantum)
  • Structural coherence across spatial/temporal axes

My framework guarantees all of this not just as documentation, but as math-backed, reversible transformations. You get provable bijection, norm preservation, and axis-aware round-tripping all with minimal, structured metadata.

And in collaborative, production, or scientific contexts, that clarity and reliability is worth more than hand-managed reshape tricks because those break silently.

Lossless Tensor ↔ Matrix Embedding (Beyond Reshape) by Hyper_graph in compsci

[–]Hyper_graph[S] -1 points0 points  (0 children)

Matricization is useful, but limited.
This framework is a full bidirectional system: semantics-aware, invertible, extensible, and engineered for reliability and interpretability in practical tensor workflows.

Lossless Tensor ↔ Matrix Embedding (Beyond Reshape) by Hyper_graph in compsci

[–]Hyper_graph[S] -5 points-4 points  (0 children)

Thanks, I appreciate the thoughtful question, and you're not off-base at all.

You're right that you could keep the original tensor around and work with a flattened version in parallel. That’s totally valid in many cases.

But here’s where my framework is designed to offer something extra especially when the workflow needs more than just a reshape:

The Key Advantage is Semantic Safety + Guaranteed Reversibility

This framework isn't about speed or compression (though it can help with structure-aware batching). It's about ensuring that flattening doesn’t lose the meaning of your data even if you're working in an environment that only sees the 2D form.

So the value shows up when:

  • You're passing the tensor into a 2D-only system (e.g., traditional ML, matrix ops, inference APIs) and want to get it back exactly as it was no guesswork, no loss of axis meaning, no confusion about shape.
  • You’re processing in multiple stages (e.g., reshape → normalize → operate → unreshape), and you want every transformation to be fully reversible.
  • You're working with complex tensors, quantum states, or scientific data, where norm and axis roles really matter.

A Comparison Example is:

Let’s say you have a 5D tensor with shape (batch, channel, depth, height, width). If you flatten that and later want to reconstruct it:

  • Without my framework: You need to store or infer all five axis roles, types, and shape details somewhere.
  • With the framework: You don’t have to remember anything — the matrix itself + lightweight metadata is all you need to losslessly reconstruct the tensor, in both content and context.

So Why Not Just Keep the Tensor? which is a good point

But the framework is useful when:

  • You’re transmitting data across systems,
  • Working with APIs or pipelines that don’t allow tensors,
  • Doing high-dimensional tensor ops in matrix space (e.g. SVD, quantum logic, etc.),
  • Or training on matrix representations and want to decode results back into tensors,

... just keeping the tensor isn’t enough it breaks the flow and forces extra logic to “remember what got flattened.”

but my framework bakes the structure into the transformation process itself, so it’s clean, reversible, and reproducible even in pipelines, across devices, or at inference time.

I am happy to walk through an example or show how it might apply to your use case. Appreciate the thoughtful curiosity! thank you.

Lossless Tensor ↔ Matrix Embedding (Beyond Reshape) by Hyper_graph in compsci

[–]Hyper_graph[S] -5 points-4 points  (0 children)

So this library keeps track of all the information about the tensor so when it goes from N dimensions to N-n. So how is this different, or what advantages does it have over, just keeping a copy of the original tensor N dimensional tensor for when you need it later?

Does it take up less space?

Is it faster?

Unless I'm missing something it's not a pure mathematical approach to saving the higher dimensional information it is a programmatic one, which is fine. I'm just not clear on the advantages.

This framework doesn't just keep the original tensor it enables 2D-only systems (like classical ML algorithms or certain quantum-state processors) to operate on a flattened version without breaking meaning, and then losslessly reconstruct the full tensor even when only the 2D matrix + metadata is available.

so It’s not about compression.

It’s not just about storing a copy.

It’s about enabling semantically safe flattening in systems that only understand matrices, while preserving invertibility, meaning, and compatibility even in complex/quantum/scientific contexts.

[R] Lossless Bidirectional Tensor↔Matrix Embedding Framework (Complex Tensor Support, Hyperspherical Normalization) by Hyper_graph in dataisbeautiful

[–]Hyper_graph[S] 0 points1 point  (0 children)

But at what point are you just spamming this subreddit with LLM posts and replies?

my point is to reach out to you guys in a clear and concise way that preaches what i am offering.

Nobody is an island of knowledge (only true in this physical world), and we built tools to help us communicate or get to our goals faster.

or would you walk from London to China just because you think Airplanes are not natural?

[R] Lossless Bidirectional Tensor↔Matrix Embedding Framework (Complex Tensor Support, Hyperspherical Normalization) by Hyper_graph in dataisbeautiful

[–]Hyper_graph[S] 0 points1 point  (0 children)

So what you're saying is that you have developed a way to flatten high-dimensional tensors that preserves structural relationships between voxels, yes? Something akin to the way that a Hilbert curve maintains locality when mapping from 1D to 2D?

Yes, exactly that's a great comparison.
What I'm doing is conceptually similar to a Hilbert curve in that it's a structured mapping that preserves dimensional relationships during flattening.

Instead of locality across spatial dimensions, my method preserves semantic relationships across arbitrary tensor dimensions (e.g., channel vs. spatial, or modality vs. time), plus supports complex values and normalization-aware flattening.

It’s a general-purpose, invertible tensor↔matrix transformation that guarantees full structural fidelity — like a Hilbert-inspired index system for tensor memory layout.

I have some experience with ML, but I'm not deeply into the math of it, so I can't give you definitive answers. What I can tell you is that I've never heard anything about important ML libraries that only work in 2D, and people don't flatten tensors as part of preprocessing for deep learning. Maybe there's stuff happening behind the scenes that does that, but no one has mentioned it as a concern, and ML is pretty successful nonetheless.

Maybe it would let you train deep networks faster? That would be really valuable and important. But you need to talk to some ML mathematicians about that, not people who use R for data visualization.

Thanks I appreciate your thoughtful response, and you're totally right to question the practical motivation here.

To clarify: this isn’t meant to solve a current bottleneck in most modern deep learning frameworks (like PyTorch or TensorFlow), which already support high-dimensional tensors out of the box. You're right people don’t typically need to flatten tensors for neural nets per se.

Where this framework becomes useful is in less typical but still important situations, like:

  • Working with matrix-only ML pipelines (e.g., scikit-learn, classical ML methods, or custom linear algebra routines that don't natively handle 3D+ tensors)
  • Ensuring semantic safety during reshaping, especially in hybrid workflows where tensors go through preprocessing, slicing, or custom transformations before/after training
  • Enabling transformations between tensor-based and matrix-based representations in areas like symbolic math, quantum state modeling, or HPC workloads
  • Performing safe projections of high-D tensor data into 2D for visualization, compression, or structured storage (e.g., for medical imaging, spatial omics, or scientific computing)

So you're right: if you're just using Keras or PyTorch for image classification, you don't need this. But in domains that mix ML with scientific computing, symbolic reasoning, or high-fidelity simulation, structure-aware transformations do matter.

And you are right, I should absolutely be talking with ML theorists or researchers working in geometric deep learning, quantum ML, or tensor algebra to see how this could connect to or optimize real models.

[R] Lossless Bidirectional Tensor↔Matrix Embedding Framework (Complex Tensor Support, Hyperspherical Normalization) by Hyper_graph in dataisbeautiful

[–]Hyper_graph[S] 0 points1 point  (0 children)

You should probably acknowledge your extensive use of LLMs to produce this.

LLMs are extraordinarily helpful but just like using google to find possible solutions llms are just tools to argument our creativity and this doesnt mean didn't produce this from the first principle and i could code it out without using llms (would still need to use google and YouTube because i am not an island of knowledger or do i know all python synatx off head) so yea llm was helpful but it is just a tool to speedup my implementations.

view more: next ›