"WIRE_INCORRECT_PAYMENT_AMOUNT (reply from remote)" payment failure

[leishman]

Issue and Steps to Reproduce

I am unable to pay an invoice to a peer I have a direct channel with because I am getting a WIRE_INCORRECT_PAYMENT_AMOUNT (reply from remote) failure when I attempt to pay. I am running a mainnet lightningd node. I have a single funded channel setup with mainnet.yalls.org. Here are the details:

cli/lightning-cli listchannels 516237:873:0
{ "channels" :
	[
		{ "source" : "023668a30d0a27304695df3fb1af55a4fb75153eac34840817cae0e6a57894fd51", "destination" : "02be5fc0b9c3f4a5155c99ba88a839e79260160eb55eec3c7aaec6b75b67166671", "short_channel_id" : "516237:873:0", "flags" : 0, "active" : true, "public" : true, "satoshis" : 8600, "last_update" : 1522648726, "base_fee_millisatoshi" : 1000, "fee_per_millionth" : 1, "delay" : 144 },
		{ "source" : "02be5fc0b9c3f4a5155c99ba88a839e79260160eb55eec3c7aaec6b75b67166671", "destination" : "023668a30d0a27304695df3fb1af55a4fb75153eac34840817cae0e6a57894fd51", "short_channel_id" : "516237:873:0", "flags" : 1, "active" : true, "public" : true, "satoshis" : 8600, "last_update" : 1522651948, "base_fee_millisatoshi" : 1000, "fee_per_millionth" : 10, "delay" : 14 } ] }

This isn't my main issue, but i'm not sure why there are two channel entries here?

I tried paying an invoice to mainnet.yalls.org:

cli/lightning-cli decodepay lnbc1500n1pdvr5udpp55r79veywjy9h2kss9rflq5zms23htdsjtulmu5d5xc9qfd686xnsdz22fjkzepqg9e8g6trd3jn5gzvd9nksmnfdenjqnn9w3mk7untypfxzumsvfjhyuneyp5kuum5vycqzysqa3aacq3egly5jy36hx4skqz43zvuysua2xky0jy7syc692vd6l50uepecn8tecge38g80rme28kufjxaslwn922p4un46s9nagkewspjkwgha
{ "currency" : "bc", "timestamp" : 1522652045, "created_at" : 1522652045, "expiry" : 3600, "payee" : "023668a30d0a27304695df3fb1af55a4fb75153eac34840817cae0e6a57894fd51", "msatoshi" : 150000, "description" : "Read Article: Lighning Network Raspberry insta", "min_final_cltv_expiry" : 144, "payment_hash" : "a0fc56648e910b755a1028d3f0505b82a375b6125f3fbe51b4360a04b747d1a7", "signature" : "304402200763dee011ca3e4a4891d5cd585802ac44ce121cea8d623e44f4098d154c6ebf022047f321ce2675e708cc4e83bc7bca8f6e2646ec3ee9954a0d793aea059f516cba" }

The response I got when I attempted to pay was:

cli/lightning-cli pay lnbc1500n1pdvr5udpp55r79veywjy9h2kss9rflq5zms23htdsjtulmu5d5xc9qfd686xnsdz22fjkzepqg9e8g6trd3jn5gzvd9nksmnfdenjqnn9w3mk7untypfxzumsvfjhyuneyp5kuum5vycqzysqa3aacq3egly5jy36hx4skqz43zvuysua2xky0jy7syc692vd6l50uepecn8tecge38g80rme28kufjxaslwn922p4un46s9nagkewspjkwgha
{ "code" : 203, "message" : "failed: WIRE_INCORRECT_PAYMENT_AMOUNT (reply from remote)", "data" : { "erring_index" : 1, "failcode" : 16400, "erring_node" : "51fd9478a5e6e0ca17088434ac3e1575fba455afb13fdf954630270a0da368366884176e940c3e8c7160024d3499539a4173661ab398a12290a23a26439fd183", "erring_channel" : "0:0:0", "failures" :
	[  ] } }

I do not know if this is an issue with yalls node or my own, but I wanted to report it because it seemed like unexpected behavior. From what I can tell this error should only trigger if I'm underpaying or grossly overpaying. I don't know why the pay command would do either. Thanks!

getinfo output

cli/lightning-cli getinfo
{ "id" : "02be5fc0b9c3f4a5155c99ba88a839e79260160eb55eec3c7aaec6b75b67166671", "port" : 9735, "address" :
	[
		{ "type" : "ipv6", "address" : "2604:5500:c225:c500:20e:c4ff:fed3:b617", "port" : 9735 } ], "version" : "v0.5.2-2016-11-21-2448-ge41d565", "blockheight" : 516243, "network" : "bitcoin" }

[ZmnSCPxj]

Your version postdates #1257, which adds the feature that we overpay slightly (within the maxfeepercent you indicate, which defaults to 0.5% if you do not indicate) in order to hide how much we are actually paying from intermediate nodes, and to hide from the payee if we are direct or indirect (if we always paid exactly when connected directly but overpaid if paying via intermediary, then the payee can figure out the payer is a direct connection depending on if it received an exact or overpaid amount).

The spec specifically allows this overpayment; perhaps yalls does not? Is there anyone who knows who yalls is and who runs it, and what implementation it uses?

Try listpayments, which should show msatoshi_sent even for failed payments, and compare to the actual target msatoshi. If it is not egregiously large (i.e. less than the maxfeepercent indicated during pay command) then the issue is at yalls side.

(we could add an override to disable overpayment though, for compatibility with noncompliant payees, but note that this reduces your privacy)

This isn't my main issue, but i'm not sure why there are two channel entries here?

One has you as source, the other has you as destination. The Lightning forwarding rates are different and are applied by each source. Notice how the channel has two different fee_per_millionth: one is 1 the other is 10. I believe lnd uses 1 as default, c-lightning uses 10. So possibly yalls is a lnd node (maybe....)?

[ZmnSCPxj]

Proposed quickfix on our side is:

1. In payalgo add a bool allow_overpay; and bool allow_reject_overpay flags to struct pay. allow_overpay starts as true, while allow_reject_overpay is taken from pay arguments and defaults to false.
2. If allow_overpay is false, do not overpay.
3. If we get WIRE_INCORRECT_PAYMENT_AMOUNT and allow_reject_overpay is true, check allow_overpay. If allow_overpay is true, clear it and retry. If it is false, propagate error.

So by default we will overpay. If the user allows the payee to reject overpay, we try to overpay, then pay exact amount if we get WIRE_INCORRECT_PAYMENT_AMOUNT after overpaying.

WIRE_INCORRECT_PAYMENT_AMOUNT is not a node-level error, and it should only be sent by the payee node, so it should not get reported to gossipd to ban the payee node.

[fixone]

I'm getting this as well on "version" : "v0.5.2-2016-11-21-2448-ge41d565" with a yalls.org payment

lnc pay -k bolt11=lntb1500n1pdvrlvjpp5ulnc3avhfrpgrjgp244d8jk88nahp48jj638z3yuetk3nrqhfp9qdz22fjkzepqg9e8g6trd3jn5gzzw4ujqcfqvajk6grpdejzqetpwfhzqar9wd6xuet5ypkk7mn90ycqzys7qrjj6ufxkcznrrp5g36q0laae3zmelv8c79y7cnvcnxnujhgagxc4pfuc52p2q7dskdtscp0tzgeg2gak8c6r5htseckhgdac8ka2qqk0ws2u maxfeepercent=1
{ "code" : 203, "message" : "failed: WIRE_INCORRECT_PAYMENT_AMOUNT (reply from remote)", "data" : { "erring_index" : 2, "failcode" : 16400, "erring_node" : "d35aff1f9aba856edd77da58e51be8d5e05624dc1a3d281892a98f6a2850c99c1fd6beee161c8cd905e464ced58b8643339eb0f0f9288c734ca8da31f3429c4c", "erring_channel" : "0:0:0", "failures" :
	[
		{ "type" : "FAIL_PAYMENT_REPLY", "erring_index" : 0, "failcode" : 4103, "erring_node" : "0b852de83cc85fd2c07426fc49b6b9605c44ad269f3e0c7d7e188a605869a660475c8857be5dfb0542e42301e4aba80ff14440cf9e75a435d2a63771f74c0958", "erring_channel" : "1288381:58:0", "route" :
			[
				{ "id" : "031055723f4c8f020ce16d2bf94efae20490b309d4ad420d548dd858a913705fe1", "channel" : "1288381:58:0", "msatoshi" : 150248, "delay" : 432 },
				{ "id" : "02ee7a1bdb3bc07ffadaad7e7678a236d557954776d372995190efe06e94ff7690", "channel" : "1288709:2:0", "msatoshi" : 150248, "delay" : 288 },
				{ "id" : "039cc950286a8fa99218283d1adc2456e0d5e81be558da77dd6e85ba9a1fff5ad3", "channel" : "1288371:18:0", "msatoshi" : 150148, "delay" : 144 } ] },
		{ "type" : "FAIL_PAYMENT_REPLY", "erring_index" : 1, "failcode" : 4103, "erring_node" : "72e93935db9203c2f801c357c492388da44b38ead511d2d5e3e162c4b22b3f94ead7d82125ae19fb6f30100e9d8d353b0a5868b3d954a3c83a7b020323e6a149", "erring_channel" : "1288815:5:0", "channel_update" : "0102bad7c3b57f468bd77c2c190a42e79958da26bb98f516fb0787756fc9d159d43d2acf0c096f83f5a29ef529bbf5edc3b25e7ce5f943b1f8fad920848630eb056743497fd7f826957108f4a30fd9cec3aeba79972084e90ead01ea33090000000013aa6f00000500005abfd7de0000000600000000000003e8000000010000000a", "route" :
			[
				{ "id" : "02943f2bb2c462e1e3d5d211d5ea384ba48d3892c457c301f8c20392db3539e972", "channel" : "1288960:112:1", "msatoshi" : 150508, "delay" : 294 },
				{ "id" : "02ee7a1bdb3bc07ffadaad7e7678a236d557954776d372995190efe06e94ff7690", "channel" : "1288815:5:0", "msatoshi" : 150506, "delay" : 288 },
				{ "id" : "039cc950286a8fa99218283d1adc2456e0d5e81be558da77dd6e85ba9a1fff5ad3", "channel" : "1288371:18:0", "msatoshi" : 150406, "delay" : 144 } ] } ] } }

[cdecker]

It seems lnd doesn't allow overpaying even on the current master version. Will flag this in the integration testing repo.

[cdecker]

Reported as cdecker/lightning-integration#15

[leishman]

Thanks for the quick response guys! Glad that this was a useful bug report. Let me know if there's any more info you need from me.

[leishman]

@ZmnSCPxj I was looking at your issue: #1089
My payment should have only been a single hop (I was paying yalls directly), so should the randomized overpayment actually have been added in there?

[ZmnSCPxj]

@leishman, I answered that question already:

(if we always paid exactly when connected directly but overpaid if paying via intermediary, then the payee can figure out the payer is a direct connection depending on if it received an exact or overpaid amount).

[ghost]

I have the same issue (WIRE_INCORRECT_PAYMENT_AMOUNT) with both Yalls and Coinpanic.com and with both I have a direct channel (http://channels.bight.nl). Yalls payments used to work flawlessly.

EDIT: I've put maxoverpayment=0; in lightnings/payalgo.c and recompiled and things are working again as is obvious from the non-zero balances for yalls and coinpanic.com (http://channels.bight.nl). Forwarded payments to https://lnmainnet.gaben.win/ are also working again :-)

[ZmnSCPxj]

It is a spec non-compliance by lnd, so I am not very motivated to add code to suppress overpayment in c-lightning (more code, is more liability). An issue is that lnd is now officially in beta and presumably mainnet nodes will upgrade only if lnd makes a new release. Hopefully lnd next release that fixes spec non-compliance occurs quickly.

[leishman]

@ZmnSCPxj I think the fix is already in the most recent release lightningnetwork/lnd@5984cbd

[cdecker]

Yep, lnd has added a fix, and as soon as nodes update this will no longer be an issue. Closing.