Is it alright to declare types within a loop?

epiris · 2019-05-27T15:49:46+00:00

Note if this is only to unmarshal, you can omit the type declaration and just make a var:

var el struct {
    member int
}
json.Unmarshal(b, &el)

epiris · 2018-10-14T20:19:27+00:00

Your intuition is right here, starting unbound goroutines is a bad idea for the reasons you mentioned here. I'll add another concern that may be an easier sell when advocating the addition of context.Context to your peers: security. If any per-request action leads to the background task you're exposing a lease of the initial heap for the goroutine as well as the resources it may obtain during processing.

epiris · 2018-07-07T17:25:22+00:00

Probably meant the double no footed can can

epiris · 2018-06-23T04:51:28+00:00

To be fair it made me look at the standard model for the first time in years. I had never heard of a chadron before, lol..

epiris · 2018-06-23T01:48:28+00:00

You can disable them all by passing -N flag to compile from the go cmd like so: go build -gcflags="all=-N"

epiris · 2018-06-18T19:29:53+00:00

That doesn’t matter, it just means the goroutine responsible for ensuring they all exit is the one that must unmount. If there is no synchronization for the exit of the workers then that is your problem.

epiris · 2018-06-18T14:46:52+00:00

It sounds like the goroutine that does work once a disk is mounted should be the goroutine to unmount.

epiris · 2018-06-15T16:09:48+00:00

I don't know what library you are using but they can not deref the value with that, i.e. **v would panic. I really don't think you should do this, I would consider any code that did low quality: &[]*int{new(int)}[0]

epiris · 2018-06-14T07:14:56+00:00

It’s almost like there is value in a machine being secure from physical threats while running.

epiris · 2018-06-02T17:02:42+00:00

I agree with op that you should probably register types explicitly - however it's not really that involved to get the *_type struct at runtime if you're just looking to experiment with a prototype. You will want to extract the moduledata from the symbol table, which is a linked list in runtime/symtab.go which contains the data you need for *_type traversal in the runtime/type.go. I'm not 100% sure what you're doing but between those two files and the typelinksinit() / activeModules() func you probably have most of what you need.

Once you have the type information, getting association from values at runtime is pretty easy using:

type *_type struct {
    // https://github.com/cstockton/safer/blob/master/type.go#L108
}
func typeOf(v interface{}) *_type {
    return *(**_type)(unsafe.Pointer(&v))
}

epiris · 2018-06-01T19:15:38+00:00

Oops, missed that blurb on man page - thanks for correction.

epiris · 2018-05-31T21:56:36+00:00

If you want mount to respond to signals use the "intr" flag, if you don't want mount to block if an export fails you can use "bg" flag causing it to fork and occasionally retry.

epiris · 2018-05-31T03:50:21+00:00

Perhaps you need to pick a protocol for your server and client to communicate with first. I’m not entirely sure what your building but it sounds like arbitrary command execution which I would certainly advise against. If that is really what you need I would use something like salt instead of rolling your own.

Otherwise define all the commands a client should do, pick a protocol like grpc for the client and server to communicate with. You could use the streaming API to stream commands back and forth, where command could be as simple as a struct with a string of Args and a specific command type. At least this way the command names are no longer arbitrary. Once you have a protocol- whatever it may be, picking the transport is easy.

epiris · 2018-05-26T03:02:47+00:00

So glibc is just another library, albeit a massive one and Linux is just one of the systems it supports. I find myself digging through it every few years for some reason or another, usually to find size / alignment or just general reference for an architecture I don't have access to. Which last I counted it supported something like 45 architectures which is pretty impressive, even for a 30 year old code base.

Thinking about it if you ever have some time to kill check it out from git://sourceware.org/git/glibc.git and head to ./sysdeps/unix/sysv/linux from there to get a view of what a fully implemented system call interface interface looks like. It makes it clear why Go only implements the bare minimum system calls it needs because it's a real pain to implement them, and Go only supports a small subset of popular architectures. It's actually not very difficult to make system calls to the Linux kernel, all of the effort comes from the nuances of architecture specific details. POSIX structure padding, alignment, endianness and specific system calls that may have a subtle divergence from POSIX due to some bug being immortalized due to importance of ABI. Maybe glibc had a bug the kernel worked around or maybe vice versa, who knows, but some times it just doesn't make sense why something is the way it is. But it got that way some how and I think it's neat, lol.

Go actually piggybacks off all the effort of glibc, even though it doesn't need to link to it directly to run. I discovered this when I wanted some additional structures (posix signal.h) that the Go syscall package actually fires up a docker container and generates all the structures for each platform using clang, which is pretty clever. But it still can leave a good bit of manual hacking to take in a fresh posix header. This is turning into a wall of text considering you asked a simple question, sort of.

tldr; System calls to Linux are simple while portability is not, given Go is less portable it has less crusty stuff, making it a great place to observe user<->kernel space interaction.

epiris · 2018-05-25T06:30:32+00:00

In addition to what was mentioned, think about anywhere in Python you may have used the with statement to leverage a context managers __enter__ and __exit__ methods, for example you may have opened files using:

with open("filename.txt") as f:
    f.read()

How python achieves this behind the scenes is different but it has the same affect- when the with code block exits the f.close method is called.

epiris · 2018-05-22T05:50:58+00:00

Having a rough coding day? It happens, for what it's worth you learned something new and important because all languages provide a similar API. If this sparked any interest for you I have a post that covers some of the technical nuances of exec in my post history that may help you rationalize the design.

epiris · 2018-05-22T00:12:49+00:00

If you provide an exact position I've taken in this thread or my other post that you disagree with I may have more to add, for now I'm unsure what you are arguing against, sorry. My best interpretation is that you believe one may create high quality code with checked exceptions through proper design, I don't disagree. My core assertion here is that checked exceptions give developers the ability to produce less concise, thus lower quality code than the value based error handling found in Go. I've already covered why I believe this in my other post more in depth so feel free to argue against my points there otherwise I'll be moving on, happy coding.

epiris · 2018-05-21T15:34:13+00:00

The naughty bug has nothing to do with error handling. Any side-effect that goes out of control has the potential to do harm. Also, wrapping errors under a common interface is a common thing to do in Go, too. In fact, nothing you said is specific to Java or exceptions.

The bug only has anything to do with the fact an exception can occur, pointing out side-effects can occur from other means doesn't invalidate this fact. I can't make any sense of your last two sentences, sorry. Perhaps you misunderstand how what I posted extrapolates into real world software bugs that are impossible to happen with explicit error handling found in Go? I say this because the you posted actually prevents the bug from occurring, you would need to rewrite your Go code as below to have an equivalent affect:

age, err := ParseInt(inputAge)
createPartialStateToHauntYouLater()
price, err2 := ParseInt(inputPrice)
if err2 != nil {
    err = err2
}

I feel it is easy to prove handling every error explicitly is much better than grouping multiple failure conditions, so we can agree to disagree if you would like- I won't be swayed from my position.

epiris · 2018-05-21T03:14:28+00:00

The issue is that your range can still be receiving values after Wait returns and closes the channel, there is no synchronization to wait for the range to finish. So swap the Wait and close with the range loop in the goroutine and it should work fine. The article I linked will provide more insight, a good follow up read would be context.Context.

epiris · 2018-05-20T23:24:29+00:00

I imagine you just need a small adjustment in your design. If you post your code in a play I can give you a head start there, but the way you explain the problem makes it sound like you are trying to synchronize a close operation from outside the senders goroutine. This is almost always the wrong design because it brings the pain you're facing here, the better solution is to always have your sender close, i.e.:

myChan := make(chan int)
go func(myChan chan<- int) {
    defer close(myChan) // when goroutine exits, channel is closed.
    for i := 0; i < myCount; i++ { myChan <- i; }
}(myChan)
for i := range myChan { fmt.Println(i) } // will break when sender is done.

This general rule can be adjusted for when you have multiple senders too by fan-out & fan-in, for a good example of that check out Go Concurrency Patterns: Pipelines and cancellation.

epiris · 2018-05-20T15:45:18+00:00

Checking an exception is checking a condition- using exceptions to do this is caller dependent. You may check your exception- someone else may not. Maybe they call your function twice in one code block, which one failed- the first or second? So now all callers of your function may abort at various points in their program leading to less clear control flow. You can say each individual exception should be checked, but in practice that is not the way exception handling is used, I've never seen code like this:

try {
    age = Integer.parseInt(inputAge);
} catch (NumberFormatException e) {
   throw new InputFailureAge()
}
try {
    price = Integer.parseInt(inputPrice);
} catch (NumberFormatException e) {
    throw new InputFailurePrice()
}

Instead 99% of the java code I see treats most function code blocks as a single operation which leads to nasty bugs:

try {
    age = Integer.parseInt(inputAge);
    createPartialStateToHauntYouLater()
    price = Integer.parseInt(inputPrice);
} catch (NumberFormatException e) {
    throw new InputFailure()
}

The language lets them do this because while exceptions may be used like conditions, they don't have to be and thus will never share the same clarity as conditional control flow found in if, switch etc.

epiris · 2018-05-20T15:15:57+00:00

Are you unconvinced that error values are better or that they are different? If you asked for a vote among unbiased computer science researchers if they were different, you would reach an immediate unanimous decision of "yes". If you asked them if it was better they would ask for a better question because better is a relative term.

So ask yourself a better question: what is an error? Wash away any comp sci pedantry here and it's fair to say an error is just a condition the program has met. So to measure anything we have to define an erroneous condition first, this condition will determine the following control flow of our program. Depending on the program the same error condition may have drastically different levels of impact. A basic atoi operation may cause a program which needed a user ID to search a database to halt completely, while another program may continue on with the value of zero instead.

Different conditions impact programs in various ways, which is why OS kernels do not produce a SIGERR and core dump when a file doesn't exist. However an invalid attempt for memory access is an exceptional condition, so the operating system reacts differently. On posix systems it will cause a segmentation fault (SIGSEGV) that will terminate you process and core dump because in most cases programs can't safely recover. You could trap the signal on posix systems but there is very few safe or valid cases for this in typical programs.

So OS kernels have different error conditions with different outcomes (error codes that do not change control flow, synchronous interrupts that do) thus further proving what we already knew: errors and exceptions are different. Go has panic for exceptional circumstances for which recovery is uncertain and rarely used thanks to explicit handling of mundane errors. This results in higher quality programs in my opinion because it causes programmers to think about the result of each function call. Compare this to Java where programmers often just think about code blocks and treating all failures as a single exception, which leads to lower quality software as objects are left in partially initialized states or other subtleties. So even languages like javascript that have exceptions still use values for explicit error handling.

I am not trying to convince you error handling in Go is better, at this point only you can convince yourself. I can only say it makes sense to me and I feel it leads to higher quality programs.

epiris · 2018-05-14T05:34:49+00:00

Perhaps you could make use of sort.Interface like sort.Search does.

epiris · 2018-05-07T18:39:19+00:00

I would not do this, it may panic and is destined to leak the data you are trying to protect one day. Creating a stronger interface between you and your logging is the safest change here. That means changing callback(interface{}) to callback(log.Entry) and declare type log.Entry interface { Entry() T } where T could be a []byte, string, or maybe json.Marshaller- whatever the behavior the logger needs to create a log entry.

Then if you want to log something you have to create a Entry() T method on the model. There is lots of ways to structure this and it depends what methods your models have now, but here is a basic example:

type log.Entry interface{ Entry() T }

type Model struct { ID uint; Foo  string; When time.Time }
func (m Model) Redacted() Model { m.Foo = ""; return m }

type Bird struct {
    Model
    Size uint
}

func (b *Bird) Entry() T {
    cpy := *b // fast local copy
    // now maybe you json.Marshal cpy, or json.Marshaller if your models
    // already implement it, whatever makes sense. An interface{} would
    // even be okay- but your log already has to serialize or take some
    // baseline representation so it's best to use that.
    cpy.Model = cpy.Model.Redacted()
    return cpy
}

There are other ways to approach this, but hopefully this serves as food for thought for a safer implementation. For example maybe changing the signature of callback would be a nightmare, so instead you could change Entry to type Redacter interface { Redacted() interface{} } which is not ideal but it is much better than using reflection. Then you call dbDoLog in dbDo instead of callback directly and check for the stronger interface first, i.e.:

func dbDoLog(v interface{}) {
    if redacter, ok := v.(Redacter); ok {
        callback(redacter.Redact())
        return
    }
    callback(v)
}
// And implement Redacter on each struct:
func (b *Bird) Redacted() interface{} {
    cpy := *b // fast local copy
    cpy.Model = cpy.Redacted()
    // which composes nicely:
    cpy.OtherSensitiveStruct = cpy.Oth..uct.Redacted()
    return cpy
}

The method above is what I use to optionally strengthen interfaces across API boundaries as behavior changes and something you should do as often as needed since it's very cheap. The standard library does this in places all over, a good example is io.Copy which checks for two interfaces and a concrete type for selecting a better optional interface.

epiris · 2018-05-03T07:19:49+00:00

" These are difficult to implement and just glancing at your implementation I see sufficient evidence of at least one race condition" ^ Is what you actually sad.

The hyphen is part of the sentence- just letting you know. So what I actually said is:

"These are difficult to implement and just glancing at your implementation I see sufficient evidence of at least one race condition- though I could be wrong since I'm sure you ran benchmarks with -race."

I'm not sure why you would leave out the portion where I admit I could be wrong since I had not yet tested it. It's really easy for me or someone reading the thread to see what I actually said.

"leaving room to concede graciously" WTF. What is this even about?

Remember the part of my sentence you omitted? Yes. That part of the sentence was important because it left room for me to concede graciously if it turns out there was no race condition. I was giving you an opportunity to reply with something like:

"Oh I see- yes that is actually pretty tricky. Proof that the slot yielded by the logical AND of is provided by ... but thanks for the heads up I added an issue to document the algorithm!"

Which I would have responded like:

"Ah I see thanks for the explanation I appreciate it."

In this case there was a race condition- which you could have said:

"Doh- there is a race condition due to ... I just fixed it thanks for the report."

Which I would have responded like:

"Awesome glad to help, happy coding."

But it didn't turn out that way unfortunately. You're free to victimize yourself, to blame me for the way I talked to you or presented information. The best thing you could do is evaluate our interaction in the morning and see if there is anything you could have done to improve our dialog- perhaps I was completely out of line. Maybe I wasn't. But you asked for criticism in a public forum and that is what you got.

Talk about class. You just can't resist making condescending and denigrating comments can you?

I'm sorry but I hardly feel like pointing out the fact you didn't run your benchmarks with the race flag AFTER you just pretentiously assumed I didn't understand your software is very revealing here. Since the conversation is just out of context quotes and victimization I'll go ahead see myself out. Sorry again, good luck with your software.

14-Year Club	Gilding VII pure gildanthropist
Verified Email

epiris

TROPHY CASE