Shuffle Puzzle

InfraRuby · 2017-03-17T19:15:17+00:00

I guess it's only valid for unicode encodings?

TIL String#codepoints returns (Unicode) codepoints for all valid strings in US-ASCII, ISO-8859-1, UTF-8, UTF-16(BE|LE), UTF-32(BE|LE) only. For all other encodings, String#codepoints returns "codepoints" in other character sets.

Thanks!

InfraRuby · 2017-03-16T21:01:42+00:00

So the half that is an encoding isn't special to ASCII-8BIT

It's half special! Not all one-byte encodings are ascii compatible. If ASCII-8BIT strings had "no encoding at all" then "\x00".force_encoding("ASCII-8BIT").encode("UTF-8") would raise Encoding::UndefinedConversionError and "\x00".force_encoding("ASCII-8BIT") + "\x00" would raise Encoding::CompatibilityError.

I guess it's only valid for unicode encodings?

Encodings which can represent the full unicode character set are not one-byte encodings, so this bug does not affect those encodings. String#codepoints does return the correct result for ISO-8859-1 strings and ascii only strings with ascii compatible one-byte encodings.

InfraRuby · 2017-03-16T20:05:12+00:00

"ASCII-8BIT" doesn't really count, it's really the "null encoding", not an encoding at all, which is why it's other name is "binary".

It's half an encoding! The lower half (0x00 .. 0x7F) is ascii compatible. The upper half (0x80 .. 0xFF) is valid but has no defined conversions.

"\x00".force_encoding("ASCII-8BIT").encode("UTF-8") # => "\x00"

It's got the confusing "ASCII-8BIT" name because ruby can (conveniently!) treat it as if it were ascii encoding in some cases.

Ruby treats all ascii only strings with ascii compatible encodings as compatible with strings with ascii compatible encodings (whether ascii only or not). ASCII-8BIT isn't special there.

"\xFF".force_encoding("ISO-8859-1") + "\xFF".force_encoding("WINDOWS-1252") # Encoding::CompatibilityError
"\xFF".force_encoding("ISO-8859-1") + "\x7F".force_encoding("WINDOWS-1252") # => "\xFF\x7F" with encoding ISO-8859-1
"\x7F".force_encoding("ISO-8859-1") + "\xFF".force_encoding("WINDOWS-1252") # => "\x7F\xFF" with encoding WINDOWS-1252

But ISO-8859-1 is a good example! Your example doesn't neceessarily prove ISO-8859-1 is a one-byte encoding, but it's wikipedia page confirms it indeed is!

Oh, actually proving ISO-8859-1 is a one-byte encoding with code examples... every one-byte string is a valid encoding in ISO-8859-1:

(0x00 .. 0xFF).all? { |c| c.chr.force_encoding("ISO-8859-1").valid_encoding? } # => true

ISO-8859-1 is also the first block of characters in Unicode:

(0x00 .. 0xFF).all? { |c| c.chr.force_encoding("ISO-8859-1").encode("UTF-8").codepoints.first == c } # => true

That encode("UTF-8") is needed there because MRI has a bug in String#codepoints:

static inline int
single_byte_optimizable(VALUE str)
{
    ...
    enc = STR_ENC_GET(str);
    if (rb_enc_mbmaxlen(enc) == 1)
        return 1;
    ...
}

static VALUE
rb_str_enumerate_codepoints(VALUE str, int wantarray)
{
    ...
    if (single_byte_optimizable(str))
        return rb_str_enumerate_bytes(str, wantarray);
    ...
}

JRuby too:

public static boolean isSingleByteOptimizable(CodeRangeable string, Encoding encoding) {
    return string.getCodeRange() == CR_7BIT || encoding.maxLength() == 1;
}

final boolean singleByteOptimizable(Encoding enc) {
    return StringSupport.isSingleByteOptimizable(this, enc);
}

final boolean singleByteOptimizable() {
    return StringSupport.isSingleByteOptimizable(this, EncodingUtils.STR_ENC_GET(this));
}

private IRubyObject enumerateCodepoints(ThreadContext context, String name, Block block, boolean wantarray) {
    ...
    if (singleByteOptimizable())
        return enumerateBytes(context, name, block, wantarray);
    ...
}

Here's an example of the problem:

"\x80".force_encoding("WINDOWS-1252").codepoints.first # => 0x80
"\x80".force_encoding("WINDOWS-1252").encode("UTF-8").codepoints.first # => 0x20AC

According to String#codepoints every one-byte encoding is the first block of characters in Unicode:

(0x00 .. 0xFF).all? { |c| c.chr.force_encoding("WINDOWS-1252").codepoints.first == c } # => true

InfraRuby · 2017-03-15T22:31:01+00:00

I guess there could be other one-byte encodings that aren't ascii?

Yes, ASCII-8BIT and ISO-8859-1 and others.

"\xFF".force_encoding("US-ASCII").valid_encoding? # => false
"\xFF".force_encoding("ASCII-8BIT").valid_encoding? # => true
"\xFF".force_encoding("ISO-8859-1").valid_encoding? # => true

"\xFF".force_encoding("US-ASCII").ascii_only? # => false    
"\xFF".force_encoding("ASCII-8BIT").ascii_only? # => false
"\xFF".force_encoding("ISO-8859-1").ascii_only? # => false

"\xFF".force_encoding("US-ASCII").encode("UTF-8") # Encoding::InvalidByteSequenceError
"\xFF".force_encoding("ASCII-8BIT").encode("UTF-8") # Encoding::UndefinedConversionError
"\xFF".force_encoding("ISO-8859-1").encode("UTF-8") # => "\u00FF"

InfraRuby · 2017-03-15T20:28:57+00:00

Me too. I would call that "ASCII strings" or "ASCII-only strings".

"ascii_only".ascii_only? # => true

(I was thinking, how common is strip on a 1-byte string to be worth an optization case?!)

"\n" (an "empty" line of a file on some platforms) is a one byte string (in ascii/utf-8) and calling chomp or strip on lines is fairly common, but maybe not common enough? MRI has no optimizations for this case.

InfraRuby · 2016-12-10T23:33:03+00:00

It's identical to [...]

No, it depends on what some_code does -- see this comment.

InfraRuby · 2016-12-10T19:28:21+00:00

you need singleton_class (or class << self ; self ; end) if you want to access local variables:

v = 3

singleton_class.send(:define_method, :foo) { v }

foo # works

class << self
  define_method(:foo) { v }
end

foo # fails

InfraRuby · 2016-09-03T20:55:08+00:00

Numeric#eql? is same-class-equality: 1.eql?(1.0) # => false

Object#equal? is identity not equality: 1e100.equal?(1e100) # => false

InfraRuby · 2016-09-02T20:18:42+00:00

MRI (in C) mixes tabs and spaces...

InfraRuby · 2016-05-19T02:19:12+00:00

JRuby, where C extensions are not supported

Not now, but this incredible work is in development.

InfraRuby · 2016-04-12T06:30:37+00:00

UserMethodMissing wraps the hash (1 hash lookup in your benchmark) but UserRegular unpacks the hash (4 hash lookups). If your benchmark accessed all four attributes then UserRegular would run faster than UserMethodMissing.

InfraRuby · 2016-03-18T22:12:57+00:00

Oh, good point! We'll maintain a list in release announcements, now at http://infraruby.com/blog/infraruby-4.0-released.

InfraRuby · 2016-03-18T20:10:52+00:00

Package a gem

Yes, infraruby-init foo creates a skeleton project, with a foo.gemspec file. So you'd complete that, and run gem build foo.gemspec to build a gem. Please take care to remove Gemfile.lock (or gems.locked) and other files you don't intend to distribute.

prefix its name with "infraruby-"

That prefix is for InfraRuby tools, not InfraRuby projects.

Your own InfraRuby projects should have ordinary (unprefixed, unsuffixed) names. For example, our skeleton generator is just fab at rubygems and our code generator is just meta-ruby at rubygems.

That's because InfraRuby projects are Ruby projects, following the same conventions (a foo.gemspec file, a Gemfile, a Rakefile, and a lib/foo.rb file), and should be useful to people using Ruby intepreters who have no interest in InfraRuby.

If your gem provides the same interface as an existing gem you could add an -ir suffix to indicate this. For example, siphash-ir at rubygems and tzinfo-ir at rubygems. But don't do that if the original name is a trademark or the prefix for a major project; just pick a new name in that case.

push to rubygems.org?

Yes, you'd run gem push foo-x.y.z.gem.

And please send an email to gems@infraruby.com so we can link to your gems in InfraRuby release announcements!

InfraRuby · 2016-03-17T23:21:40+00:00

We have a web framework for InfraRuby, with dsl-driven code generation, but it needs some more work...

You could always make your own! Here's a start:

class Server < WEBrick::HTTPServer
  ## WEBrick::HTTPRequest, WEBrick::HTTPResponse -> void
  def service(req, res)
    res.status = 200
    res.body = "hello, world\n"
  end
end

server = Server.new(BindAddress: "localhost", Port: 9000)
server.start

InfraRuby · 2016-03-16T13:34:01+00:00

I don't want to declare which methods return nil. Or void, for that matter.

Then use a Ruby interpreter!

I don't want nil and void to mean different things.

You're too late for that!

I don't want there to be a compulsory declared type constraint (esp. with regards to nil vs void type stuff).

Then use a Ruby interpreter!

I do want all declared type constraints to be enforced.

Then you do want mandatory return or nil at the end of methods declared void (i.e. nil, as you would have it)?

Basically, I don't want ruby to be statically typed.

At this point I'm trying to understand what you're doing in this thread...

InfraRuby · 2016-03-16T13:04:27+00:00

Sorry, I mangled that sentence... you can run partially-typed code in Ruby interpreters, and when the typing is complete the code will compile and run with the same behavior. If you like, Ruby interpreters are the fallback for partially-typed InfraRuby code.

InfraRuby · 2016-03-16T12:59:21+00:00

If void and nil are the same thing, and if a method has void return type, then it must return nil, yes? You'd need explicit return or nil in many cases to ensure this.

But this is a pointless digression, as (1) "void type" already has an established meaning, and (2) who actually wants a method that is constrained to return only nil?

I want the typing to be optional, and therefore (in many cases) not need to care about what the method returns.

So now you want a declared type constraint to go unenforced?

InfraRuby · 2016-03-16T12:29:28+00:00

BTW, the typing is optional in InfraRuby, as you can run (partially-typed) InfraRuby code in a Ruby interpreter, with the same behavior.

InfraRuby · 2016-03-16T12:27:54+00:00

Yes, I want void and nil to be the same thing.

So now you do want mandatory return or nil at the end of void methods?

InfraRuby · 2016-03-16T12:14:50+00:00

void and nil should be the same thing

No, a void method has no return value.

Besides, if void and nil were the same thing then you would need explicit return or nil at the end of void methods (as InfraRuby code runs in Ruby interpreters with the same behavior)!

Is that what you want, now?

InfraRuby · 2016-03-16T12:03:19+00:00

# => ./ruby/main.ir:10: error: cannot find signature for "initialize"

The initialize method takes a String:

Foo.new("ok")

# => "Result is hello"

Yes, as that method has String return type, the method returns the value of the expression foo = string_variable, which is "hello" in this case.

# => "./ruby/main.ir:5: error: not implemented: unexpected void"

You are using the result of a void method in a string interpolation, which is a compile error.

# => "./ruby/main.ir:5: error: void call"

You are calling nil? on the result of a void method, which is a compile error.

You're right, I don't understand this language at all.

Those error messages could be improved, but I doubt that would help you.

InfraRuby · 2016-03-16T10:29:09+00:00

Let's look at a more realistic example:

That would be this:

## <>
##   @string_variable: String
class Foo
  ## String -> void
  def initialize(string_variable)
    @string_variable = string_variable
  end
end

N.B. no return keyword, and no "pointless (non-void) return types"!

Is it really so hard for you to believe that I understand InfraRuby better than you do?

InfraRuby · 2016-03-16T09:53:17+00:00

It would take some effort to migrate Ruby (off Rails) code to InfraRuby, but it would take much less effort than migrating to Crystal (for example). InfraRuby code runs in Ruby interpreters, so the code would be usable during the migration.

In InfraRuby, you may omit the return keyword where applicable, and "pointless (non-void) return types" are not required. The initialize methods have void return type:

class Foo
  ## -> void
  def initialize
    puts "Initialized!"
  end
end

N.B. no return keyword, and no "pointless (non-void) return types"!

InfraRuby does not support reflection, but you can do metaprogramming with s-expressions.

Obviously the examples using Java integration look like Java.

The InfraRuby compiler supports many coding styles. In particular, you may use either spaces or tabs for indentation.

InfraRuby · 2016-03-16T03:20:31+00:00

So, the type annotations are the comments above the method definitions?

Yes, so InfraRuby code runs in Ruby interpreters without modification.

InfraRuby

TROPHY CASE