Add integer formatting code for fixed-width signed arithmetic (2's complement)

[rhettinger]

BPO	30571
Nosy	@rhettinger, @pitrou, @ericvsmith, @skrah, @serhiy-storchaka, @lisroach, @csabella

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/lisroach'
closed_at = None
created_at = <Date 2017-06-05.04:03:26.700>
labels = ['interpreter-core', 'type-feature', '3.7']
title = "Add integer formatting code for fixed-width signed arithmetic (2's complement)"
updated_at = <Date 2017-06-13.03:25:21.157>
user = 'https://github.com/rhettinger'

bugs.python.org fields:

activity = <Date 2017-06-13.03:25:21.157>
actor = 'lisroach'
assignee = 'lisroach'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2017-06-05.04:03:26.700>
creator = 'rhettinger'
dependencies = []
files = []
hgrepos = []
issue_num = 30571
keywords = []
message_count = 8.0
messages = ['295162', '295749', '295752', '295753', '295754', '295793', '295805', '295843']
nosy_count = 8.0
nosy_names = ['rhettinger', 'talin', 'pitrou', 'eric.smith', 'skrah', 'serhiy.storchaka', 'lisroach', 'cheryl.sabella']
pr_nums = []
priority = 'low'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue30571'
versions = ['Python 3.7']

Linked PRs

• gh-74756: support precision field for integer formatting types #131926

[rhettinger]

The current 'b' formatting code in inconvenient for emulating, demonstrating, and teaching two's complement arithmetic. The problem is that negative inputs are always formatted with a minus sign. I propose that some formatting code be provided for fixed-width display where the leading bit is a sign bit.

For example, if code were a capital 'B' and the total width were 8-bits:

    >>> x = -12
    >>> format(12, '08B')
    '11110100'

Currently, to achieve the same effect, one of the following is used:

    >>> format(x if x >= 0 else x + 2**8, '08b')
    '11110100'

or

    >>> format(x & (2**8 - 1), '08b')
    '11110100'

For values outside the valid range, perhaps a ValueError could be raised:

    >>> format(-200, '08B')
    Traceback (most recent call last):
    ...
    ValueError:  Expected value in range -128 <= x < 128, not -200

I'm not sure what the right code should be. The idea of capital 'B' is attractive, but we already have a different relationship between 'X' and 'x'. There could also be a modifier symbol such as '!' in '!8b'.

[mdickinson]

-1 from me. Using format(x % 2**8, '08b') seems both short enough to be easy to use, and long enough to remind one that there's something a little bit unnatural going on here, given that two's complement isn't something that makes sense for arbitrary-sized integers.

[serhiy-storchaka]

Concur with Mark. Similar issues already were raised multiple times on mailing lists and the conclusion is that explicit wrapping integers to specific range is better. Different behavior for integers out of range is needed in different applications.

[rhettinger]

This is a recurring need in my teaching of Python to hardware engineers. The whole point of having a binary format code is to show which bits are set. For negative values, that need is not being served by the current option. And it makes it awkward when modeling the effects of bitwise operations on signed numbers.

I can understand a -1 if you think this is fundamentally broken, but if you're just saying that you've never personally needed this or casually dismissing the awkwardness of the usual workarounds, then it seems like a meaningless vote that blows-off my proposal which is based on real-world classroom experiences.

The idea is that modifier (such as "!" would require a width argument (the signed representation only make sense in fixed width concepts as every assembly language programmer knows). Its presence would also imply the "0".

Serhiy, I believe you've either read different maillist posts than I have or that you're misinterpreting this as a proposal for a fixed width integer type with automatic wrap-around and truncation (full emulation of a register). This proposal is for display formatting only. IMO, it remedies a deficiency where the current option is more for our convenience (not wanting to deal with the width) rather than for the user's convenience where the user wants to see which bits are set rather than seeing a minus sign (which is both obvious and useless).

Also, when a senior coredev presents a proposal, I expect that it will be greeted with a little more open mindedness and not instantly shot down as if I have no idea what I'm talking about.

[serhiy-storchaka]

The idea is that modifier (such as "!" would require a width argument (the signed representation only make sense in fixed width concepts as every assembly language programmer knows). Its presence would also imply the "0".

I would use the precision for this. Truncating the number of digits in integer representation is similar to truncating the number of characters for strings.

But actually I don't think we should add such feature in the core. It has very limited application. This can be implemented as a third-part library.

Actually I think the stdlib needs something like bitarray and/or bitset. The specially purposed types that allow testing and modifying separate bits and bit ranges.

[skrah]

To expand on what Mark said: If the proposed format code implicitly
gives these results ...

>>> format(12 % 2**8, '08b')
'00001100'
>>> format(-12 % 2**8, '08b')
'11110100'

..., would students not expect that these two operations have the same
result?

>>> 0b11110100 | 0b00001100
252
>>> 
>>> -12 | 12
-4

[pitrou]

The format code space is already crowded, and it isn't immediately obvious that '08B' will format the 2's complement representation of the number, as opposed to '08b' which prepends a sign bit. Not to mention that, as Raymond remarked, we cannot use the same convention for hex formatting where 'X' already has a different meaning.

Instead, how about adding an optional argument to bin()? And then we can also do the same for hex().

By the way, I do find annoying the default behaviour of prepending a sign bit on hexadecimal and binary displays (but I never use binary displays, so my annoyance is really with hexadecimal displays).

[lisroach]

I can't see the drawback of being able to format two's complement easily, isn't it more common to represent negative binary numbers in two's complement form? I find the - sign a little odd myself.

I agree with the idea of using "!" with the precision to represent width.

[skirpichev]

What's status of this issue?

It seems, this idea got no wide support from core developers. Maybe it would be better to discuss that on https://discuss.python.org/c/ideas/6 ?

[rhettinger]

I didn't expect "wide support". The group of people who work with binary representations of number is a small group. People who have never needed a programmer's calculator are unlikely to even recognize that the current 'b' format code is almost never useful for negative numbers.

When a user has a real world need to see the a binary representation of a negative number, 99% of the time they want to see the two's complement. Pretty much the only reason to display a negative number in binary is to see how it is stored.

FWIW for small features, our main criterion is usefulness. "Wide support" is too high of a bar; generally "some support" and no "fierce opposition" is sufficient. Also, it tends to be easier if the door has already been opened and we're fixing the utility of a feature. In this case, we already have a 'b' format code and a bin() builtin so we're really only considering whether negative numbers can be handled usefully (thereby remedying a design flaw in the 'b' format code — the 'B' is what 'b' could have been has negative inputs been thought of from a user's point of view).

This particular feature is not often needed, but when you do need it, it is really inconvenient to not have it.

[ericvsmith]

I'm not sure what the best way to implement this in the format spec is, but me and my HP-16C (and now my Swissmicros DM16C) would definitely find this useful.

[skirpichev]

generally (1) "some support" and no (2) "fierce opposition" is sufficient.

That's exactly the case, sorry for being imprecise. So far, things looks more like (2).

When a user has a real world need to see the a binary representation of a negative number, 99% of the time they want to see the two's complement.

Maybe (and same for "x" code). Though, e.g. gmp_printf() shows signed integer for Z type specifier (mpz_t big integer) and x/o formatting types.

But probably this ship is sailed. Or not?

thereby remedying a design flaw in the 'b' format code — the 'B' is what 'b' could have been has negative inputs been thought of from a user's point of view

Maybe we can fix this flaw instead? (BTW, now C has 'B' format type.)

As Serhiy suggested above, I would use precision field to specify fixed-width. Recently, there was a thread, suggested to add precision support for new-style string formatting of integer types: https://discuss.python.org/t/80760/. Then, for "b", "x" and "X" formatting types - we can show two's complements either with a specified precision or "big enough" for a given negative value.

Backward-compatible version of above - show two's complements iff precision is specified.

This particular feature is not often needed, but when you do need it, it is really inconvenient to not have it.

Most strong argument against from this issue thread was (for me): "The format code space is already crowded".

I don't expect this feature will be discoverable for people (unless being a default for b/x formatting types). BTW, just recently while debugging I quickly ended up with a code snippet like in Mark's reply. This was easy and obvious.

[skirpichev]

pr is available to review: #131926

Twos complement used here for non-decimal integer formatting types iff precision was specified.

[jb2170]

PEP in progress for Precision and Modulo-Precision format specifiers. I'll keep posting. @skirpichev I'd hold off on the PR for now.

[skirpichev]

Ok, it seems PEP going to be sponsored: python/peps#4365. Thus, I'm closing my pr.

@jb2170, please either revert part, related to this issue or ensure Raymond is ok with your design. He was asking for formatting in two's complement representation, not for a shortcut val % 2**n. And his motivation was teaching hardware arithmetic. Which means (well, for me), that we should communicate somehow to user, that val is out of the specified range (by precision n).