Thursday, November 6, 2014

PostgreSQL gotcha of the week, week 45

Something happened earlier this week (or maybe it was last week, I forget), and I was quite perplexed for a good 30 seconds, so I thought I'd try and make this problem a bit more well-known.  Consider the following schema:

CREATE TABLE blacklist (
  person text
);

INSERT INTO blacklist VALUES ('badguy');

CREATE TABLE orders (
  orderid serial PRIMARY KEY,
  person text,
  amount numeric
);

CREATE FUNCTION is_blacklisted(_person text)
  RETURNS boolean
  STABLE
AS $$
BEGIN
  RETURN EXISTS (SELECT 1 FROM blacklist
                 WHERE person = _person);
END
$$ LANGUAGE plpgsql;

CREATE FUNCTION new_order(_person text, _amount numeric)
  RETURNS bigint
AS $$
DECLARE
_orderid bigint;
BEGIN
  IF is_blacklisted(_person) THEN
    RAISE EXCEPTION 'person has been blacklisted';
  END IF;

  INSERT INTO orders (person, amount)
    VALUES (_person, _amount)
  RETURNING orderid INTO _orderid;
  RETURN _orderid;
END
$$ LANGUAGE plpgsql;

Now everything is running smoothly:

=# select new_order('goodguy', 100.0);
 new_order 
-----------
         1
(1 row)

=# select new_order('badguy', 100.0);
ERROR:  person has been blacklisted

But something's wrong with the blacklist table: it doesn't tell you who blacklisted the person or why.  You also happen to know that you added all of the existing people on the blacklist, so you want to have them reflect that.  You run the following ALTER TABLEs:

ALTER TABLE blacklist
  ADD COLUMN reason text,
  -- use DEFAULT to assign values to existing rows
  ADD COLUMN blacklisted_by text
    NOT NULL DEFAULT 'you';

-- and then drop it
ALTER TABLE blacklist
  ALTER COLUMN blacklisted_by
    DROP DEFAULT;

And everything seems to be working nicely.  However, a few days later you realize that "badguy" stole all your money by getting through a couple of fake orders.  How did that happen?

The answer is a bit complicated, but the gist of it is that if a STABLE function accesses the table for the first time in a transaction while a table-rewriting operation (such as the ALTER TABLE we used) is going on concurrently, it will see an empty table, instead of either the old or the new version of the data.  is_blacklisted() returned FALSE because of this, and "badguy" happened to be able to create an order.  This caveat applies to ALL SERIALIZATION MODES; not even READ COMMITTED users are safe.  However, SQL language functions don't seem to have this problem in more recent versions of postgres.

This really questions the safety of table-rewriting operations in an application heavy on PL/PgSQL (or some other procedural languages).  I know we're banning them internally.

7 comments:

  1. This is why the VOLATILE decorator exists. :)

    ReplyDelete
    Replies
    1. By that logic you would end up with only IMMUTABLE and VOLATILE functions in your database. That doesn't sound very attractive.

      Delete
  2. I consider this a bug. It should either use the table based on the data before the table rewrite or wait for the table rewrite to conclude. Stable should be nothing more than a hint to the planner about the reuse of the results.

    ReplyDelete
  3. Seems to correlate with my experience with COPY TO:

    http://www.postgresql.org/message-id/alpine.LNX.2.11.1411031751320.29849@titan.int.lan.stealer.net
    http://www.postgresql.org/message-id/alpine.LNX.2.11.1411041946500.26161@titan.int.lan.stealer.net

    ReplyDelete
  4. I would regard this as a bug. Have you reported it?

    ReplyDelete
  5. I can't really claim this to be a bug. The behaviour is documented:

    For functions written in SQL or in any of the standard procedural languages, there is a second important property determined by the volatility category, namely the visibility of any data changes that have been made by the SQL command that is calling the function. A VOLATILE function will see such changes, a STABLE or IMMUTABLE function will not. This behavior is implemented using the snapshotting behavior of MVCC (see Chapter 13): STABLE and IMMUTABLE functions use a snapshot established as of the start of the calling query, whereas VOLATILE functions obtain a fresh snapshot at the start of each query they execute.

    Because of this snapshotting behavior, a function containing only SELECT commands can safely be marked STABLE, even if it selects from tables that might be undergoing modifications by concurrent queries. PostgreSQL will execute all commands of a STABLE function using the snapshot established for the calling query, and so it will see a fixed view of the database throughout that query.

    (http://www.postgresql.org/docs/9.3/static/xfunc-volatility.html)

    This is just a slightly less obvious side effect of the snapshotting behaviour.

    ReplyDelete